-- Logs begin at Wed 2022-08-24 05:23:27 UTC, end at Wed 2022-08-24 07:08:53 UTC. -- Aug 24 05:56:06.093337 dv-cinder-854269-1-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Aug 24 05:56:06.103385 dv-cinder-854269-1-iscsi devstack@keystone.service[81840]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Aug 24 05:56:06.105488 dv-cinder-854269-1-iscsi devstack@keystone.service[81840]: *** Starting uWSGI 2.0.18-debian (64bit) on [Wed Aug 24 05:56:06 2022] *** Aug 24 05:56:06.105488 dv-cinder-854269-1-iscsi devstack@keystone.service[81840]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Aug 24 05:56:06.105488 dv-cinder-854269-1-iscsi devstack@keystone.service[81840]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Aug 24 05:56:06.105488 dv-cinder-854269-1-iscsi devstack@keystone.service[81840]: nodename: dv-cinder-854269-1-iscsi Aug 24 05:56:06.105488 dv-cinder-854269-1-iscsi devstack@keystone.service[81840]: machine: x86_64 Aug 24 05:56:06.105488 dv-cinder-854269-1-iscsi devstack@keystone.service[81840]: clock source: unix Aug 24 05:56:06.105488 dv-cinder-854269-1-iscsi devstack@keystone.service[81840]: pcre jit disabled Aug 24 05:56:06.105488 dv-cinder-854269-1-iscsi devstack@keystone.service[81840]: detected number of CPU cores: 4 Aug 24 05:56:06.105488 dv-cinder-854269-1-iscsi devstack@keystone.service[81840]: current working directory: / Aug 24 05:56:06.105488 dv-cinder-854269-1-iscsi devstack@keystone.service[81840]: detected binary path: /usr/bin/uwsgi-core Aug 24 05:56:06.105488 dv-cinder-854269-1-iscsi devstack@keystone.service[81840]: your processes number limit is 47680 Aug 24 05:56:06.105488 dv-cinder-854269-1-iscsi devstack@keystone.service[81840]: your memory page size is 4096 bytes Aug 24 05:56:06.105488 dv-cinder-854269-1-iscsi devstack@keystone.service[81840]: detected max file descriptor number: 2048 Aug 24 05:56:06.105488 dv-cinder-854269-1-iscsi devstack@keystone.service[81840]: lock engine: pthread robust mutexes Aug 24 05:56:06.106232 dv-cinder-854269-1-iscsi devstack@keystone.service[81840]: thunder lock: enabled Aug 24 05:56:06.106232 dv-cinder-854269-1-iscsi devstack@keystone.service[81840]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Aug 24 05:56:06.106232 dv-cinder-854269-1-iscsi devstack@keystone.service[81840]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Aug 24 05:56:06.144644 dv-cinder-854269-1-iscsi devstack@keystone.service[81840]: Python main interpreter initialized at 0x5635f32779c0 Aug 24 05:56:06.144644 dv-cinder-854269-1-iscsi devstack@keystone.service[81840]: python threads support enabled Aug 24 05:56:06.144644 dv-cinder-854269-1-iscsi devstack@keystone.service[81840]: your server socket listen backlog is limited to 100 connections Aug 24 05:56:06.144644 dv-cinder-854269-1-iscsi devstack@keystone.service[81840]: your mercy for graceful operations on workers is 90 seconds Aug 24 05:56:06.145002 dv-cinder-854269-1-iscsi devstack@keystone.service[81840]: mapped 403077 bytes (393 KB) for 2 cores Aug 24 05:56:06.145078 dv-cinder-854269-1-iscsi devstack@keystone.service[81840]: *** Operational MODE: preforking *** Aug 24 05:56:06.145335 dv-cinder-854269-1-iscsi devstack@keystone.service[81840]: *** uWSGI is running in multiple interpreter mode *** Aug 24 05:56:06.145335 dv-cinder-854269-1-iscsi devstack@keystone.service[81840]: spawned uWSGI master process (pid: 81840) Aug 24 05:56:06.146141 dv-cinder-854269-1-iscsi systemd[1]: Started Devstack devstack@keystone.service. Aug 24 05:56:06.150053 dv-cinder-854269-1-iscsi devstack@keystone.service[81840]: spawned uWSGI worker 1 (pid: 81851, cores: 1) Aug 24 05:56:06.150053 dv-cinder-854269-1-iscsi devstack@keystone.service[81840]: spawned uWSGI worker 2 (pid: 81852, cores: 1) Aug 24 05:56:06.150053 dv-cinder-854269-1-iscsi devstack@keystone.service[81840]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Aug 24 05:56:10.491814 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 24 05:56:10.491814 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: from cryptography.utils import int_from_bytes Aug 24 05:56:10.492657 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 24 05:56:10.492657 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: from cryptography.utils import int_from_bytes Aug 24 05:56:10.492771 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 24 05:56:10.492771 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: from cryptography.utils import int_from_bytes Aug 24 05:56:10.492771 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 24 05:56:10.492771 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: from cryptography.utils import int_from_bytes Aug 24 05:56:10.549394 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 24 05:56:10.549623 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 24 05:56:10.549708 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 24 05:56:10.549821 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] command line args: [] {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 24 05:56:10.549913 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 24 05:56:10.550029 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] command line args: [] {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 24 05:56:10.550107 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 24 05:56:10.550261 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] ================================================================================ {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 24 05:56:10.550345 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 24 05:56:10.550470 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] ================================================================================ {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 24 05:56:10.550557 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] admin_token = **** {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:56:10.550729 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] admin_token = **** {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:56:10.550807 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] config_dir = [] {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:56:10.550925 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:56:10.551008 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] config_dir = [] {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:56:10.551109 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] config_source = [] {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:56:10.551197 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:56:10.551274 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] debug = True {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:56:10.551346 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] config_source = [] {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:56:10.551501 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:56:10.551753 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] debug = True {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:56:10.551753 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:56:10.551904 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:56:10.551966 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:56:10.552022 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:56:10.552179 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:56:10.552289 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:56:10.552399 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:56:10.552518 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:56:10.552635 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:56:10.552749 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] list_limit = None {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:56:10.552845 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] list_limit = None {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:56:10.552950 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] log_config_append = None {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:56:10.553046 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] log_config_append = None {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:56:10.553140 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:56:10.553239 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:56:10.553348 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] log_dir = None {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:56:10.553449 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] log_dir = None {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:56:10.553554 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] log_file = None {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:56:10.553662 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] log_file = None {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:56:10.553765 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:56:10.553866 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:56:10.553951 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:56:10.554068 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:56:10.554157 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:56:10.554230 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:56:10.554344 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:56:10.554463 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:56:10.554586 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:56:10.554708 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:56:10.554806 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:56:10.554917 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:56:10.554985 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:56:10.555110 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:56:10.555216 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:56:10.555307 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:56:10.555414 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:56:10.555534 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:56:10.555647 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:56:10.555732 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:56:10.555879 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:56:10.555958 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:56:10.556109 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:56:10.556187 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:56:10.556354 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:56:10.556481 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:56:10.556645 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:56:10.556752 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:56:10.556888 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:56:10.557000 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:56:10.557378 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.62/identity {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:56:10.557506 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.62/identity {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:56:10.557628 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] publish_errors = False {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:56:10.557741 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] publish_errors = False {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:56:10.557900 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:56:10.558023 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:56:10.558127 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:56:10.558237 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:56:10.558354 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:56:10.558467 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:56:10.558590 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:56:10.558720 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:56:10.558845 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:56:10.558961 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:56:10.559073 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] standard_threads = False {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:56:10.559195 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] standard_threads = False {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:56:10.559321 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:56:10.559436 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:56:10.559548 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:56:10.559660 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:56:10.559777 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:56:10.559899 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:56:10.560014 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] use_journal = False {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:56:10.560126 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] use_journal = False {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:56:10.560237 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] use_json = False {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:56:10.560355 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] use_json = False {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:56:10.560469 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] use_stderr = False {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:56:10.560618 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] use_stderr = False {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:56:10.560735 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] use_syslog = False {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:56:10.560848 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] use_syslog = False {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:56:10.560968 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:56:10.561090 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:56:10.561210 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.561397 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.561514 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.561634 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.561754 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.561869 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.561981 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.562106 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.562227 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.562227 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.562448 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.562560 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.562681 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.562802 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.562923 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.563073 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.563197 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.563311 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.563436 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.563551 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.563670 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.563789 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.563908 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.564022 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.564142 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.564257 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.564257 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.564487 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.564639 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.564760 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.564882 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.564998 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.565117 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.565233 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.565351 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.565465 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.565585 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.565714 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.565887 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.566004 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.566125 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.566245 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.566367 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.566482 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.566595 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.566595 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.566806 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.566919 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.567603 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.567723 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.567843 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.567958 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.568071 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.568191 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.568317 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.568432 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.568558 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.568673 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.568797 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.568918 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.569041 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.569162 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.569276 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.569389 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.569389 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.569601 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.569722 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.569843 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.569969 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.570083 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.570196 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.570315 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.570438 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.570553 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.570666 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.570778 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.570875 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.570976 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.571086 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.571189 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.571284 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.571379 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.571506 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.571616 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.571720 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.571817 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.571932 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.572046 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.572161 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.572282 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.572397 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.572549 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.572667 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.572781 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.572906 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.573034 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.573162 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.573284 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.573407 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.573528 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.573648 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.573751 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.573862 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.573984 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.574107 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.574222 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.574335 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.574448 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.574570 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.574686 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.574801 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.574915 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.575029 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.575131 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.575250 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.575366 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.575469 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.575580 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.575695 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.575808 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.575929 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.576051 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.576174 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.576289 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.576403 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.576560 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.576687 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.576816 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.576938 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.577053 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.577175 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.577290 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.577404 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.577525 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.577641 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.577755 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.577869 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.577983 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.578096 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.578218 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.578333 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.578453 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.578574 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.578689 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.578802 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.578995 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.579112 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.579226 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.579339 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.579459 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.579574 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.579699 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.579814 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.579930 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.580043 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.580157 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.580271 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.580401 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.580558 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.580678 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.580792 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.580907 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.581023 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.581127 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.581230 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.581333 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.581430 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.581533 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.581630 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.581726 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.581829 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.581927 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.582023 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.582119 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.582215 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.582309 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.582412 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.582517 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.582613 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.582747 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.582861 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.582861 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.583058 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.583155 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.583250 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.583355 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.583451 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.583553 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.583662 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.583759 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.583861 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.583956 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.584052 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.584148 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.584260 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.584357 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.584357 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.584594 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.584594 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.585318 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.593449 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.593571 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.593674 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.593867 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] auth.external = None {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.593867 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.593998 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.593998 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.594194 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.594268 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.594339 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.594339 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] auth.password = None {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.594473 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.594551 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] auth.token = None {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.594649 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.594736 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.594815 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] auth.external = None {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.594887 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.594957 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.595027 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.595097 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.595179 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.595253 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.595323 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.595394 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] auth.password = None {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.595464 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.595534 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] auth.token = None {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.595613 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.595686 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.595756 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] credential.caching = True {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.595827 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.595915 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.596037 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.596119 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.596193 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.596263 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.596334 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.596405 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.596483 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.596607 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.596680 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.596751 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.596822 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] credential.caching = True {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.596893 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.596976 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.597050 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.597122 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.597200 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.597274 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.597344 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.597426 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.597500 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.597571 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.597642 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.597720 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.597792 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.597871 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.597944 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.598015 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.598086 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.598157 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.598246 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] federation.caching = True {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.598398 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.598474 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.598546 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.598624 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.598697 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.598768 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.598846 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.598920 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.598991 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.599062 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.599133 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.599203 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.599281 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] federation.caching = True {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.599354 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.599426 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.599536 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.599612 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.599688 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.599761 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.599834 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.599905 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.599976 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.600046 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.600046 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.600181 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] identity.caching = True {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.600254 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.600325 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.600396 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.600466 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.600466 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.600639 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.600714 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.600785 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.600856 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.600926 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.600996 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.601081 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] identity.caching = True {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.601154 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.601224 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.601294 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.601365 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.601435 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.601518 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.601596 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.601667 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.601737 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.601808 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.601878 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.601956 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.602029 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.602100 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.602170 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.602247 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.602327 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.602400 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.602471 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.602541 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.602612 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.602683 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.602683 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.602819 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.602890 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.602961 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.603031 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.603102 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.603102 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.603239 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.603310 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.603380 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.603451 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.603521 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.603599 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.603672 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.603742 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.603813 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.603883 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.603952 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.604030 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.604103 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.604225 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.604302 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.604373 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.604444 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.604542 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.604618 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.604689 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.604761 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.604831 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.604902 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.604980 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.605054 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.605125 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.605195 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.605266 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.605266 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.605407 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.605480 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.605551 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.605621 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.605692 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.605762 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.605840 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.605913 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.605991 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.606064 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.606135 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.606205 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.606282 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.606355 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.606426 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.606497 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.606567 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.606637 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.606760 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.606844 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.606916 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.606987 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.606987 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.607115 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.607195 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.607268 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.607338 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.607409 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.607480 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] ldap.user = None {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.607550 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.607628 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.607701 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.607772 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.607843 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.607962 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.608044 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.608211 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.608294 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.608521 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.608612 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.608765 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.608847 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.609007 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.609090 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.609250 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] ldap.user = None {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.609331 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.609486 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.609572 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.609727 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.609808 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.609965 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.610052 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.610200 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.610280 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.610448 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.610528 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.610695 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.610774 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.610929 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.611019 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.611163 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.611243 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.611402 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.611483 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.611632 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.611722 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.611867 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.611946 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.612140 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.612277 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.612398 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.612567 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.612690 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.612806 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.612922 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.613036 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.613151 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.613275 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.613393 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.613510 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.613625 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.613740 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.613873 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.614001 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.614119 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.614225 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.614339 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.614454 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.614570 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.614693 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.614810 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.614926 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.615041 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.615156 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.615271 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.615462 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.615582 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.615697 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.615813 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.615928 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.616043 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.616167 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.616284 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] resource.caching = True {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.616401 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.616533 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.616653 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.616780 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.616907 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.617031 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.617148 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.617264 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.617380 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.617495 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.617696 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.617832 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.617949 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.618065 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.618181 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.618296 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.618458 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] resource.caching = True {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.618458 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.618693 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.618810 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.618810 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.619027 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] role.caching = True {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.619151 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.619273 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] role.driver = sql {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.619378 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.619476 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.619573 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.619670 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.619777 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.619876 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.619974 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.620070 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.620165 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.620261 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.620366 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] role.caching = True {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.620464 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.620615 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] role.driver = sql {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.620729 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.620852 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.620959 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.625673 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.625785 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.625866 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.625940 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.626011 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.626083 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.626168 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.626243 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.626315 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.626386 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.626457 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.626536 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.626610 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.626681 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.626752 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.626823 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.626894 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.626973 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.627051 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.627122 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.627193 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.627263 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.627333 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.627412 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.627485 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.627556 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.627556 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.627683 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.627753 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.627836 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.627910 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.627981 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.628052 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.628123 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.628194 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.628274 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.628347 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.628418 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.628488 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.628585 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.628656 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.628656 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.628800 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.628871 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.628941 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.629011 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.629082 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.629160 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.629303 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.629378 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] token.caching = True {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.629450 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.630048 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.630122 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.630203 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.630277 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.630349 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.630420 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.630490 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.630562 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.630645 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.630645 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.630776 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.630847 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.630918 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.630988 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.631067 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.631140 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.631211 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] token.caching = True {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.631282 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.631353 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.631430 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.631503 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.631582 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.631653 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.631724 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.631794 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.631878 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.631951 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.632023 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.632093 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.632164 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.632234 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.632313 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.632386 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.632457 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] database.connection = **** {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.632561 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.632644 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.632718 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.632789 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.632860 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.632938 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.633011 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.633081 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.633152 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.633222 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.633292 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.633370 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.633443 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.633514 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.633585 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.633688 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.633867 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.633957 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.634134 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] database.connection = **** {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.634223 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.634384 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.634466 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.634677 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.634771 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.634922 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.634999 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.635134 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.635215 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.635374 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.635455 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.635616 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.635698 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.635856 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.635942 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.636097 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.636177 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.636258 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 24 05:56:10.636332 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.636596 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.636846 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.637085 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.637336 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.637574 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.637842 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.638078 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.638321 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.638558 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:56:10.638731 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 24 05:56:10.666464 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81851) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 05:56:10.666694 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81851) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 05:56:10.666917 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81851) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 05:56:10.667899 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81852) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 05:56:10.668159 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81852) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 05:56:10.668394 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81852) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 05:56:10.675917 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81851) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 05:56:10.677205 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81852) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 05:56:10.704880 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81851) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 05:56:10.705117 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81851) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 05:56:10.708617 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81852) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 05:56:10.708617 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81852) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 05:56:10.762226 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81851) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 05:56:10.762589 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81852) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 05:56:10.811752 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81851) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 05:56:10.811983 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81851) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 05:56:10.813282 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81852) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 05:56:10.813533 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81852) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 05:56:10.978446 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81851) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 05:56:10.978674 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81851) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 05:56:10.978896 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81851) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 05:56:10.979192 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81851) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 05:56:10.979411 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81851) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 05:56:10.979624 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81851) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 05:56:10.979736 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81852) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 05:56:10.979971 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81852) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 05:56:10.980197 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81852) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 05:56:10.980419 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81852) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 05:56:10.980664 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81852) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 05:56:10.980884 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81852) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 05:56:11.003333 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81851) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 05:56:11.009558 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81851) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 05:56:11.009558 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81851) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 05:56:11.009558 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81851) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 05:56:11.009558 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81851) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 05:56:11.009558 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81851) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 05:56:11.009558 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81851) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 05:56:11.010595 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81852) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 05:56:11.010839 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81852) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 05:56:11.011070 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81852) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 05:56:11.011302 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81852) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 05:56:11.011525 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81852) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 05:56:11.011779 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81852) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 05:56:11.011966 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81852) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 05:56:11.020180 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81851) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 05:56:11.022382 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81852) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 05:56:11.032669 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81851) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 05:56:11.032899 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81851) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 05:56:11.033118 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81851) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 05:56:11.033200 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81852) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 05:56:11.033334 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81851) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 05:56:11.033416 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81852) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 05:56:11.033545 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81851) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 05:56:11.033629 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81852) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 05:56:11.033755 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81851) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 05:56:11.033837 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81852) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 05:56:11.033966 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81851) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 05:56:11.034057 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81852) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 05:56:11.034260 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81852) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 05:56:11.034477 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81852) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 05:56:11.050917 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81851) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:56:11.051123 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81851) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 24 05:56:11.051302 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:56:11.051998 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81851) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:56:11.052082 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 24 05:56:11.052197 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81851) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 24 05:56:11.052348 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:56:11.052471 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81851) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:56:11.052595 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 24 05:56:11.052756 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81851) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:56:11.052837 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:56:11.053095 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:56:11.053261 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81851) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:56:11.053521 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81851) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:56:11.053614 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:56:11.053861 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:56:11.062972 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81851) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:56:11.063248 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81851) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:56:11.063331 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:56:11.063520 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81851) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:56:11.063636 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:56:11.063803 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81851) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:56:11.063919 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:56:11.064208 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:56:11.071456 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81851) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 05:56:11.071900 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81852) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 05:56:11.074360 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81851) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 05:56:11.074643 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81851) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:56:11.074846 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81852) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 05:56:11.074931 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81851) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:56:11.084473 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:56:11.084586 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81851) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:56:11.084663 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:56:11.084736 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81851) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:56:11.084808 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:56:11.084894 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81851) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:56:11.084971 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:56:11.085043 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81851) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:56:11.085123 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:56:11.085197 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81851) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:56:11.085278 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:56:11.085353 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81851) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:56:11.085425 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:56:11.085497 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81851) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:56:11.085569 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:56:11.085641 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81851) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:56:11.085716 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:56:11.085716 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:56:11.099826 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81851) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:56:11.100597 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:56:11.101308 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81851) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 05:56:11.101592 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81851) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:56:11.102081 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81852) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 05:56:11.102373 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:56:11.106284 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81851) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 05:56:11.106566 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81851) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:56:11.106847 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81851) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:56:11.107101 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81852) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 05:56:11.107414 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:56:11.107706 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:56:11.113215 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81851) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 05:56:11.113498 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81851) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:56:11.120221 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81852) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 05:56:11.120221 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:56:11.120359 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81851) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 05:56:11.120359 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81851) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:56:11.120359 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81851) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:56:11.120359 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81851) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:56:11.120647 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81852) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 05:56:11.120722 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81851) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:56:11.120784 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:56:11.120855 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81851) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:56:11.120935 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:56:11.121010 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81851) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:56:11.121083 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:56:11.121162 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81851) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:56:11.121237 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:56:11.121237 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:56:11.121237 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:56:11.121237 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:56:11.137753 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81851) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:56:11.138032 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81851) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:56:11.138878 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:56:11.139166 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:56:11.141916 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81851) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 05:56:11.143061 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81852) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 05:56:11.145237 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81851) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:56:11.145536 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81851) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:56:11.146344 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:56:11.146648 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:56:11.149779 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81851) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 05:56:11.150918 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81852) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 05:56:11.152910 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81851) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 05:56:11.154128 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81852) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 05:56:11.156128 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81851) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:56:11.156438 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81851) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:56:11.156767 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81851) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:56:11.157062 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81851) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:56:11.157355 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81851) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:56:11.157449 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:56:11.157644 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81851) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:56:11.157727 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:56:11.157978 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:56:11.158290 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:56:11.158578 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:56:11.158875 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:56:11.174330 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81851) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 05:56:11.174629 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81851) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:56:11.174901 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81851) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:56:11.175170 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81851) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:56:11.175727 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81852) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 05:56:11.176033 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:56:11.176325 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:56:11.177160 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:56:11.182620 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81851) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:56:11.184085 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:56:11.184179 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81851) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:56:11.184411 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81851) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:56:11.185655 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:56:11.185939 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:56:11.187082 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81851) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 05:56:11.187366 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81851) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:56:11.187658 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81851) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:56:11.187951 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81851) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:56:11.188228 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81851) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:56:11.188626 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81852) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 05:56:11.188917 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:56:11.189254 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:56:11.189551 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:56:11.189836 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:56:11.199755 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81851) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 05:56:11.200040 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81851) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:56:11.200326 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81851) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:56:11.200642 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81851) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:56:11.200928 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81851) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:56:11.201208 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81851) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:56:11.201290 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81852) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 05:56:11.201497 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81851) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:56:11.201585 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:56:11.201821 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:56:11.202108 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:56:11.202396 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:56:11.202676 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:56:11.203244 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:56:11.218003 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81851) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 05:56:11.218822 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81852) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 05:56:11.221057 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81851) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 05:56:11.221934 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81852) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 05:56:11.224467 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81851) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:56:11.225430 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:56:11.263066 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81851) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:56:11.263158 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:56:11.263231 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81851) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 05:56:11.263231 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81851) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:56:11.263231 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81851) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:56:11.263415 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81852) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 05:56:11.263415 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:56:11.263415 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:56:11.263671 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81851) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:56:11.263746 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:56:11.263818 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81851) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 05:56:11.263900 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81852) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 05:56:11.263975 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81851) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:56:11.263975 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81851) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:56:11.263975 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81851) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:56:11.263975 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81851) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:56:11.264224 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:56:11.264224 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:56:11.264224 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:56:11.264224 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:56:11.264553 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81851) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 05:56:11.264553 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81851) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:56:11.264553 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81851) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:56:11.264749 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81852) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 05:56:11.264749 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:56:11.264749 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:56:11.264946 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81851) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 05:56:11.264946 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81851) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:56:11.264946 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81851) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:56:11.264946 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81851) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:56:11.264946 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81851) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:56:11.264946 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81851) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:56:11.265317 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81852) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 05:56:11.265390 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81851) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:56:11.265463 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:56:11.265525 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81851) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:56:11.265596 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:56:11.265677 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81851) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:56:11.265752 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:56:11.265824 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81851) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:56:11.265896 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:56:11.265969 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81851) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:56:11.266048 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:56:11.266123 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81851) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:56:11.266195 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:56:11.266267 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81851) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:56:11.266339 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:56:11.266419 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81851) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:56:11.266502 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:56:11.266502 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:56:11.266502 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:56:11.266502 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:56:11.266502 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:56:11.266502 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:56:11.294431 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x5635f32779c0 pid: 81851 (default app) Aug 24 05:56:11.296516 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x5635f32779c0 pid: 81852 (default app) Aug 24 05:56:11.298077 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e064566-08d9-4c81-b51d-a25019859362 None None] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:11.298364 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e064566-08d9-4c81-b51d-a25019859362 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:11.298634 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e064566-08d9-4c81-b51d-a25019859362 None None] PATH_INFO: `/v3` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:11.303309 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1/1] 10.222.0.62 () {50 vars in 791 bytes} [Wed Aug 24 05:56:11 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:56:18.889758 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6091df7-774d-4d9e-95e2-2ee1dbb6492e None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:18.890291 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6091df7-774d-4d9e-95e2-2ee1dbb6492e None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:18.890291 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6091df7-774d-4d9e-95e2-2ee1dbb6492e None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:18.894552 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1/2] 10.222.0.62 () {54 vars in 927 bytes} [Wed Aug 24 05:56:18 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:56:18.901546 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684dbe27-9bd7-493d-82c3-e3cc7e6c06e2 None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:18.901932 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684dbe27-9bd7-493d-82c3-e3cc7e6c06e2 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:18.902088 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684dbe27-9bd7-493d-82c3-e3cc7e6c06e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:18.972057 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: INFO dbcounter [None req-684dbe27-9bd7-493d-82c3-e3cc7e6c06e2 None None] Registered counter for database keystone Aug 24 05:56:18.992663 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG oslo_db.sqlalchemy.engines [None req-684dbe27-9bd7-493d-82c3-e3cc7e6c06e2 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81851) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 24 05:56:18.998735 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writer thread running {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 24 05:56:19.177990 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG passlib.handlers.bcrypt [None req-684dbe27-9bd7-493d-82c3-e3cc7e6c06e2 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=81851) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 24 05:56:19.178401 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG passlib.handlers.bcrypt [None req-684dbe27-9bd7-493d-82c3-e3cc7e6c06e2 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81851) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 24 05:56:19.232337 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-684dbe27-9bd7-493d-82c3-e3cc7e6c06e2 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:56:19.233901 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-684dbe27-9bd7-493d-82c3-e3cc7e6c06e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:56:19.363923 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 2/3] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Aug 24 05:56:18 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 463 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 24 05:56:19.828629 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6091df7-774d-4d9e-95e2-2ee1dbb6492e None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:56:19.896907 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: INFO dbcounter [None req-b6091df7-774d-4d9e-95e2-2ee1dbb6492e None None] Registered counter for database keystone Aug 24 05:56:19.916217 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG oslo_db.sqlalchemy.engines [None req-b6091df7-774d-4d9e-95e2-2ee1dbb6492e None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81852) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 24 05:56:19.922540 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writer thread running {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 24 05:56:20.041036 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6091df7-774d-4d9e-95e2-2ee1dbb6492e None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:56:20.045153 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed43db3b-fe8d-4d25-9ab3-518a2c6dce9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:56:20.046044 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed43db3b-fe8d-4d25-9ab3-518a2c6dce9c admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:20.046320 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed43db3b-fe8d-4d25-9ab3-518a2c6dce9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:20.046590 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed43db3b-fe8d-4d25-9ab3-518a2c6dce9c admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:20.054507 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed43db3b-fe8d-4d25-9ab3-518a2c6dce9c admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:56:20.105794 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: WARNING keystone.common.rbac_enforcer.enforcer [None req-ed43db3b-fe8d-4d25-9ab3-518a2c6dce9c admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 24 05:56:20.106135 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed43db3b-fe8d-4d25-9ab3-518a2c6dce9c admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:56:20.115346 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 2/4] 10.222.0.62 () {58 vars in 1324 bytes} [Wed Aug 24 05:56:19 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 287 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 24 05:56:20.121352 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684dbe27-9bd7-493d-82c3-e3cc7e6c06e2 None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:56:20.135432 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684dbe27-9bd7-493d-82c3-e3cc7e6c06e2 None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:56:20.139759 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad5da35-18b8-41af-b61e-1b8e0a8f83a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:56:20.140904 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad5da35-18b8-41af-b61e-1b8e0a8f83a8 admin admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:20.141189 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad5da35-18b8-41af-b61e-1b8e0a8f83a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:20.141448 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad5da35-18b8-41af-b61e-1b8e0a8f83a8 admin admin] PATH_INFO: `/v3/projects/632e93ce840f4853b64b4d659536e045` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:20.142987 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad5da35-18b8-41af-b61e-1b8e0a8f83a8 admin admin] RBAC: Authorizing `identity:get_project(project_id=632e93ce840f4853b64b4d659536e045)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:56:20.200307 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: WARNING keystone.common.rbac_enforcer.enforcer [None req-cad5da35-18b8-41af-b61e-1b8e0a8f83a8 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 24 05:56:20.200740 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad5da35-18b8-41af-b61e-1b8e0a8f83a8 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:56:20.205223 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 3/5] 10.222.0.62 () {56 vars in 1207 bytes} [Wed Aug 24 05:56:20 2022] GET /identity/v3/projects/632e93ce840f4853b64b4d659536e045 => generated 344 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:56:22.076135 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7614f703-a3ff-4ef7-a723-f7666d397ee2 None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:22.076696 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7614f703-a3ff-4ef7-a723-f7666d397ee2 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:22.076696 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7614f703-a3ff-4ef7-a723-f7666d397ee2 None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:22.080719 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 3/6] 10.222.0.62 () {54 vars in 927 bytes} [Wed Aug 24 05:56:22 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:56:22.086613 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29828c86-41b4-484f-94ce-6ca555a40574 None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:22.087034 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29828c86-41b4-484f-94ce-6ca555a40574 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:22.087170 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29828c86-41b4-484f-94ce-6ca555a40574 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:22.107446 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-29828c86-41b4-484f-94ce-6ca555a40574 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:56:22.109013 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-29828c86-41b4-484f-94ce-6ca555a40574 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:56:22.116744 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 4/7] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Aug 24 05:56:22 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 24 05:56:22.385699 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7614f703-a3ff-4ef7-a723-f7666d397ee2 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:56:22.398254 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7614f703-a3ff-4ef7-a723-f7666d397ee2 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:56:22.402461 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3db90e-b338-497d-8b7c-781c8a3cc3c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:56:22.403056 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3db90e-b338-497d-8b7c-781c8a3cc3c7 admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:22.403337 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3db90e-b338-497d-8b7c-781c8a3cc3c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:22.403612 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3db90e-b338-497d-8b7c-781c8a3cc3c7 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:22.411725 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3db90e-b338-497d-8b7c-781c8a3cc3c7 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:56:22.413542 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3db90e-b338-497d-8b7c-781c8a3cc3c7 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:56:22.422523 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 4/8] 10.222.0.62 () {58 vars in 1324 bytes} [Wed Aug 24 05:56:22 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 05:56:22.427674 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29828c86-41b4-484f-94ce-6ca555a40574 None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:56:22.436649 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29828c86-41b4-484f-94ce-6ca555a40574 None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:56:22.441097 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15d934f-1046-4689-9e34-683210b3f26a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:56:22.442211 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15d934f-1046-4689-9e34-683210b3f26a admin admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:22.442542 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15d934f-1046-4689-9e34-683210b3f26a admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:22.442819 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15d934f-1046-4689-9e34-683210b3f26a admin admin] PATH_INFO: `/v3/users/8b2c3a376af245f2a0e347e2f08885a5` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:22.444128 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15d934f-1046-4689-9e34-683210b3f26a admin admin] RBAC: Authorizing `identity:get_user(user_id=8b2c3a376af245f2a0e347e2f08885a5)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:56:22.446001 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15d934f-1046-4689-9e34-683210b3f26a admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:56:22.447512 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 5/9] 10.222.0.62 () {56 vars in 1198 bytes} [Wed Aug 24 05:56:22 2022] GET /identity/v3/users/8b2c3a376af245f2a0e347e2f08885a5 => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:56:24.401343 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311bae11-0c95-4ba8-baea-df88d6c3f817 None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:24.439924 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311bae11-0c95-4ba8-baea-df88d6c3f817 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:24.439924 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311bae11-0c95-4ba8-baea-df88d6c3f817 None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:24.439924 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 5/10] 10.222.0.62 () {54 vars in 927 bytes} [Wed Aug 24 05:56:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:56:24.440185 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac54bd17-f2ca-4656-b231-39e448804403 None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:24.440185 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac54bd17-f2ca-4656-b231-39e448804403 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:24.440185 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac54bd17-f2ca-4656-b231-39e448804403 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:24.440185 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-ac54bd17-f2ca-4656-b231-39e448804403 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:56:24.440185 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-ac54bd17-f2ca-4656-b231-39e448804403 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:56:24.464311 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38261502-6ced-488c-83b3-329b8fc872e7 None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:24.464621 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38261502-6ced-488c-83b3-329b8fc872e7 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:24.464897 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38261502-6ced-488c-83b3-329b8fc872e7 None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:24.466232 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 6/11] 10.222.0.62 () {54 vars in 927 bytes} [Wed Aug 24 05:56:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 05:56:24.471249 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbd7166-bfef-4533-bfde-09aeae46453d None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:24.471529 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbd7166-bfef-4533-bfde-09aeae46453d None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:24.471800 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbd7166-bfef-4533-bfde-09aeae46453d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:24.510372 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 6/12] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Aug 24 05:56:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 05:56:24.513372 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG passlib.handlers.bcrypt [None req-8cbd7166-bfef-4533-bfde-09aeae46453d None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=81852) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 24 05:56:24.513795 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG passlib.handlers.bcrypt [None req-8cbd7166-bfef-4533-bfde-09aeae46453d None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81852) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 24 05:56:24.559808 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-8cbd7166-bfef-4533-bfde-09aeae46453d None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:56:24.561414 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-8cbd7166-bfef-4533-bfde-09aeae46453d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:56:24.657202 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 7/13] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Aug 24 05:56:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 05:56:24.781369 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac54bd17-f2ca-4656-b231-39e448804403 None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:56:24.806903 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:56:24.885918 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac54bd17-f2ca-4656-b231-39e448804403 None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:56:24.918624 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9061815-f278-43ac-b5af-f7082d6e1e59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:56:24.919262 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9061815-f278-43ac-b5af-f7082d6e1e59 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:24.919541 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9061815-f278-43ac-b5af-f7082d6e1e59 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:24.919815 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9061815-f278-43ac-b5af-f7082d6e1e59 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:24.923219 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbd7166-bfef-4533-bfde-09aeae46453d None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:56:24.932861 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9061815-f278-43ac-b5af-f7082d6e1e59 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:56:24.941662 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application [None req-a9061815-f278-43ac-b5af-f7082d6e1e59 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 05:56:24.941662 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:56:24.941662 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:56:24.941662 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:56:24.941662 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:56:24.941662 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:56:24.941662 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:56:24.941662 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:56:24.941662 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:56:24.941662 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:56:24.941662 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:56:24.941662 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:56:24.941662 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:56:24.941662 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:56:24.941662 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:56:24.941662 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:56:24.942781 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:56:24.942781 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:56:24.942781 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:56:24.942781 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:56:24.942781 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:56:24.942781 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:56:24.942781 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:56:24.942781 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:56:24.942781 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:56:24.942781 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:56:24.942781 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:56:24.942781 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application with Lock( Aug 24 05:56:24.942781 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:56:24.942781 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:56:24.942781 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:56:24.942781 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:56:24.943647 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:56:24.943647 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:56:24.943647 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:56:24.943647 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:56:24.943647 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:56:24.943647 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:56:24.943647 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:56:24.943647 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:56:24.943647 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:56:24.943647 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:56:24.943647 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 05:56:24.943647 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Aug 24 05:56:24.943647 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 7/14] 10.222.0.62 () {56 vars in 1096 bytes} [Wed Aug 24 05:56:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:56:24.948327 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9061815-f278-43ac-b5af-f7082d6e1e59 None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:56:25.025491 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbd7166-bfef-4533-bfde-09aeae46453d None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:56:25.027778 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:56:25.029925 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:56:25.048212 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9061815-f278-43ac-b5af-f7082d6e1e59 None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:56:25.062863 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d14e77e-e130-4a3d-b1ca-70abc26036c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:56:25.068151 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d14e77e-e130-4a3d-b1ca-70abc26036c7 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:25.068151 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d14e77e-e130-4a3d-b1ca-70abc26036c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:25.068151 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d14e77e-e130-4a3d-b1ca-70abc26036c7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:25.082713 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636f241b-355f-4011-b44f-1766bb3fa435 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:56:25.083320 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636f241b-355f-4011-b44f-1766bb3fa435 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:25.083602 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636f241b-355f-4011-b44f-1766bb3fa435 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:25.083874 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636f241b-355f-4011-b44f-1766bb3fa435 None admin] PATH_INFO: `/v3/roles` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:25.084556 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636f241b-355f-4011-b44f-1766bb3fa435 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:56:25.084863 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636f241b-355f-4011-b44f-1766bb3fa435 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:56:25.086256 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636f241b-355f-4011-b44f-1766bb3fa435 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:56:25.098403 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 8/15] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Aug 24 05:56:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:56:25.103876 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636f241b-355f-4011-b44f-1766bb3fa435 None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:56:25.133171 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d14e77e-e130-4a3d-b1ca-70abc26036c7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:56:25.134703 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d14e77e-e130-4a3d-b1ca-70abc26036c7 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:56:25.200830 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636f241b-355f-4011-b44f-1766bb3fa435 None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:56:25.231574 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 8/16] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Aug 24 05:56:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 24 05:56:25.235690 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d145a8-9891-4247-bcf9-cef3409851c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:56:25.236395 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d145a8-9891-4247-bcf9-cef3409851c9 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:25.236722 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d145a8-9891-4247-bcf9-cef3409851c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:25.236996 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d145a8-9891-4247-bcf9-cef3409851c9 None admin] PATH_INFO: `/v3/users/8b2c3a376af245f2a0e347e2f08885a5` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:25.237132 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d14e77e-e130-4a3d-b1ca-70abc26036c7 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:56:25.238119 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d145a8-9891-4247-bcf9-cef3409851c9 None admin] RBAC: Authorizing `identity:get_user(user_id=8b2c3a376af245f2a0e347e2f08885a5)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:56:25.239585 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d145a8-9891-4247-bcf9-cef3409851c9 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:56:25.239894 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:56:25.241184 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 9/17] 10.222.0.62 () {56 vars in 1177 bytes} [Wed Aug 24 05:56:25 2022] GET /identity/v3/users/8b2c3a376af245f2a0e347e2f08885a5 => generated 247 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:56:25.246442 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d145a8-9891-4247-bcf9-cef3409851c9 None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:56:25.260413 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:56:25.334486 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d14e77e-e130-4a3d-b1ca-70abc26036c7 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:56:25.344227 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d145a8-9891-4247-bcf9-cef3409851c9 None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:56:25.367230 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864e0b9f-faa3-4328-8cb7-e1fbae97bd54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:56:25.368071 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864e0b9f-faa3-4328-8cb7-e1fbae97bd54 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:25.368347 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864e0b9f-faa3-4328-8cb7-e1fbae97bd54 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:25.368653 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864e0b9f-faa3-4328-8cb7-e1fbae97bd54 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:25.380316 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7899637f-f572-489b-9969-999d20c2da19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:56:25.381372 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7899637f-f572-489b-9969-999d20c2da19 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:25.381774 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7899637f-f572-489b-9969-999d20c2da19 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:25.382130 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7899637f-f572-489b-9969-999d20c2da19 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:25.383482 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7899637f-f572-489b-9969-999d20c2da19 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:56:25.385141 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7899637f-f572-489b-9969-999d20c2da19 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:56:25.388193 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 10/18] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Aug 24 05:56:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:56:25.391741 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864e0b9f-faa3-4328-8cb7-e1fbae97bd54 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:56:25.393596 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864e0b9f-faa3-4328-8cb7-e1fbae97bd54 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:56:25.393914 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7899637f-f572-489b-9969-999d20c2da19 None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:56:25.412692 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-864e0b9f-faa3-4328-8cb7-e1fbae97bd54 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:56:25.412692 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:56:25.412692 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:56:25.412692 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:56:25.412692 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:56:25.412692 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:56:25.412692 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:56:25.412692 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:56:25.412692 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:56:25.412692 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:56:25.412692 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:56:25.412692 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:56:25.412692 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 05:56:25.412692 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 05:56:25.412692 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 05:56:25.412692 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 05:56:25.413925 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:56:25.413925 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:56:25.413925 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:56:25.413925 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:56:25.413925 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:56:25.413925 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:56:25.413925 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:56:25.413925 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 24 05:56:25.413925 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:56:25.413925 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:56:25.413925 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:56:25.413925 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:56:25.413925 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:56:25.413925 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:56:25.413925 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:56:25.413925 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:56:25.413925 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 05:56:25.414713 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 05:56:25.414713 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:56:25.414713 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 24 05:56:25.414713 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 9/19] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Aug 24 05:56:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 05:56:25.419479 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864e0b9f-faa3-4328-8cb7-e1fbae97bd54 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:56:25.475661 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:56:25.476194 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:56:25.494362 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7899637f-f572-489b-9969-999d20c2da19 None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:56:25.519017 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864e0b9f-faa3-4328-8cb7-e1fbae97bd54 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:56:25.528569 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878b1c91-e0b3-4c01-a7b5-427053b60ca7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:56:25.529181 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878b1c91-e0b3-4c01-a7b5-427053b60ca7 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:25.529464 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878b1c91-e0b3-4c01-a7b5-427053b60ca7 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:25.529746 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878b1c91-e0b3-4c01-a7b5-427053b60ca7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:25.530462 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878b1c91-e0b3-4c01-a7b5-427053b60ca7 None admin] RBAC: Adding query filter params (role.id=49e47cd157d743f6905f1db07ec878a7, scope.domain.id=default, user.id=8b2c3a376af245f2a0e347e2f08885a5) {{(pid=81851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:56:25.530770 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878b1c91-e0b3-4c01-a7b5-427053b60ca7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:56:25.532229 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878b1c91-e0b3-4c01-a7b5-427053b60ca7 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:56:25.540661 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 11/20] 10.222.0.62 () {56 vars in 1322 bytes} [Wed Aug 24 05:56:25 2022] GET /identity/v3/role_assignments?user.id=8b2c3a376af245f2a0e347e2f08885a5&scope.domain.id=default&role.id=49e47cd157d743f6905f1db07ec878a7 => generated 233 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:56:25.552966 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae63add-70c7-41e9-ab4e-6fa56bac2f33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:56:25.553606 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae63add-70c7-41e9-ab4e-6fa56bac2f33 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:25.553884 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae63add-70c7-41e9-ab4e-6fa56bac2f33 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:25.554153 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae63add-70c7-41e9-ab4e-6fa56bac2f33 None admin] PATH_INFO: `/v3/domains` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:25.554810 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae63add-70c7-41e9-ab4e-6fa56bac2f33 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:56:25.555112 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae63add-70c7-41e9-ab4e-6fa56bac2f33 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:56:25.556550 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae63add-70c7-41e9-ab4e-6fa56bac2f33 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:56:25.577625 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 10/21] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Aug 24 05:56:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:56:27.603267 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4966d20-2ae0-4a42-b64a-f76dcf912ad9 None None] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:27.603787 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4966d20-2ae0-4a42-b64a-f76dcf912ad9 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:27.603890 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4966d20-2ae0-4a42-b64a-f76dcf912ad9 None None] PATH_INFO: `/` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:27.605463 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 12/22] 10.222.0.62 () {54 vars in 927 bytes} [Wed Aug 24 05:56:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:56:27.610662 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbf8c7b-f771-4d44-a80d-46de2b9bc629 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:27.611080 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbf8c7b-f771-4d44-a80d-46de2b9bc629 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:27.611265 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbf8c7b-f771-4d44-a80d-46de2b9bc629 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:27.637723 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-0dbf8c7b-f771-4d44-a80d-46de2b9bc629 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:56:27.638921 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-0dbf8c7b-f771-4d44-a80d-46de2b9bc629 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:56:27.745881 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 11/23] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Aug 24 05:56:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 05:56:27.813704 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ba6c5e-ac33-4ca6-9921-2ad906de6930 None None] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:27.814016 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ba6c5e-ac33-4ca6-9921-2ad906de6930 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:27.814263 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ba6c5e-ac33-4ca6-9921-2ad906de6930 None None] PATH_INFO: `/` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:27.815596 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 13/24] 10.222.0.62 () {54 vars in 927 bytes} [Wed Aug 24 05:56:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:56:27.825531 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3185ca-aeba-4e32-b567-9729eae6989e None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:27.825825 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3185ca-aeba-4e32-b567-9729eae6989e None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:27.826102 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3185ca-aeba-4e32-b567-9729eae6989e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:27.849474 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-ef3185ca-aeba-4e32-b567-9729eae6989e None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:56:27.850669 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-ef3185ca-aeba-4e32-b567-9729eae6989e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:56:27.895127 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8389b9ff-2358-4e84-a080-6342c69ff9e5 None None] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:27.895301 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8389b9ff-2358-4e84-a080-6342c69ff9e5 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:27.895584 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8389b9ff-2358-4e84-a080-6342c69ff9e5 None None] PATH_INFO: `/` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:27.896967 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 14/25] 10.222.0.62 () {54 vars in 927 bytes} [Wed Aug 24 05:56:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 05:56:27.902178 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0a7ebb-3f5f-4687-90ce-c0bbd01eee4f None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:27.902459 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0a7ebb-3f5f-4687-90ce-c0bbd01eee4f None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:27.902729 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0a7ebb-3f5f-4687-90ce-c0bbd01eee4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:27.912943 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:56:27.935495 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-4f0a7ebb-3f5f-4687-90ce-c0bbd01eee4f None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:56:27.941853 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-4f0a7ebb-3f5f-4687-90ce-c0bbd01eee4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:56:27.969527 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 12/26] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Aug 24 05:56:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 05:56:28.054756 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 15/27] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Aug 24 05:56:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 05:56:28.193401 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3185ca-aeba-4e32-b567-9729eae6989e None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:56:28.310884 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3185ca-aeba-4e32-b567-9729eae6989e None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:56:28.344752 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ead0f7-289c-4b40-9910-276a0e376e2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:56:28.345391 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ead0f7-289c-4b40-9910-276a0e376e2d None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:28.345675 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ead0f7-289c-4b40-9910-276a0e376e2d None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:28.345951 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ead0f7-289c-4b40-9910-276a0e376e2d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:28.357870 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ead0f7-289c-4b40-9910-276a0e376e2d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:56:28.364179 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-f3ead0f7-289c-4b40-9910-276a0e376e2d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 05:56:28.364179 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:56:28.364179 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:56:28.364179 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:56:28.364179 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:56:28.364179 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:56:28.364179 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:56:28.364179 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:56:28.364179 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:56:28.364179 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:56:28.364179 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:56:28.364179 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:56:28.364179 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:56:28.364179 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:56:28.364179 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:56:28.364179 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:56:28.365052 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:56:28.365052 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:56:28.365052 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:56:28.365052 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:56:28.365052 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:56:28.365052 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:56:28.365052 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:56:28.365052 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:56:28.365052 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:56:28.365052 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:56:28.365052 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:56:28.365052 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 24 05:56:28.365052 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:56:28.365052 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:56:28.365052 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:56:28.365052 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:56:28.366230 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:56:28.366230 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:56:28.366230 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:56:28.366230 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:56:28.366230 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:56:28.366230 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:56:28.366230 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:56:28.366230 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:56:28.366230 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:56:28.366230 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:56:28.366230 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 05:56:28.366230 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 24 05:56:28.366230 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 13/28] 10.222.0.62 () {56 vars in 1096 bytes} [Wed Aug 24 05:56:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:56:28.371248 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0a7ebb-3f5f-4687-90ce-c0bbd01eee4f None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:56:28.420979 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ead0f7-289c-4b40-9910-276a0e376e2d None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:56:28.473145 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:56:28.477142 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:56:28.520739 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0a7ebb-3f5f-4687-90ce-c0bbd01eee4f None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:56:28.585925 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001bb1dd-24d0-4948-b4dc-4150e78f99c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:56:28.586541 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001bb1dd-24d0-4948-b4dc-4150e78f99c3 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:28.586861 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001bb1dd-24d0-4948-b4dc-4150e78f99c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:28.587151 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001bb1dd-24d0-4948-b4dc-4150e78f99c3 None admin] PATH_INFO: `/v3/roles` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:28.587818 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001bb1dd-24d0-4948-b4dc-4150e78f99c3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:56:28.588122 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001bb1dd-24d0-4948-b4dc-4150e78f99c3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:56:28.589545 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001bb1dd-24d0-4948-b4dc-4150e78f99c3 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:56:28.594195 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ead0f7-289c-4b40-9910-276a0e376e2d None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:56:28.611043 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 16/29] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Aug 24 05:56:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:56:28.611904 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001bb1dd-24d0-4948-b4dc-4150e78f99c3 None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:56:28.637435 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7d6919-3574-41f1-b8ac-175d135c968d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:56:28.638819 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7d6919-3574-41f1-b8ac-175d135c968d None admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:28.639244 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7d6919-3574-41f1-b8ac-175d135c968d None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:28.640040 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7d6919-3574-41f1-b8ac-175d135c968d None admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:28.641184 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7d6919-3574-41f1-b8ac-175d135c968d None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:56:28.643183 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7d6919-3574-41f1-b8ac-175d135c968d None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:56:28.735282 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001bb1dd-24d0-4948-b4dc-4150e78f99c3 None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:56:28.773408 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a6f484-afa1-4943-8e3a-325481685614 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:56:28.774101 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a6f484-afa1-4943-8e3a-325481685614 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:28.774435 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a6f484-afa1-4943-8e3a-325481685614 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:28.774711 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a6f484-afa1-4943-8e3a-325481685614 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:28.778703 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:56:28.789364 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a6f484-afa1-4943-8e3a-325481685614 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:56:28.790925 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a6f484-afa1-4943-8e3a-325481685614 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:56:28.807967 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application [None req-26a6f484-afa1-4943-8e3a-325481685614 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:56:28.807967 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:56:28.807967 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:56:28.807967 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:56:28.807967 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:56:28.807967 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:56:28.807967 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:56:28.807967 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:56:28.807967 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:56:28.807967 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:56:28.807967 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:56:28.807967 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:56:28.807967 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 05:56:28.807967 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 05:56:28.807967 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 05:56:28.807967 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 05:56:28.808909 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:56:28.808909 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:56:28.808909 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:56:28.808909 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:56:28.808909 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:56:28.808909 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:56:28.808909 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:56:28.808909 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application with Lock( Aug 24 05:56:28.808909 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:56:28.808909 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:56:28.808909 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:56:28.808909 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:56:28.808909 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:56:28.808909 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:56:28.808909 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:56:28.808909 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:56:28.808909 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 05:56:28.809689 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 05:56:28.809689 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:56:28.809689 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Aug 24 05:56:28.814405 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 17/30] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Aug 24 05:56:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:56:28.815324 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a6f484-afa1-4943-8e3a-325481685614 None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:56:28.883280 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 14/31] 10.222.0.62 () {60 vars in 1131 bytes} [Wed Aug 24 05:56:28 2022] POST /identity/v3/roles => generated 219 bytes in 463 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 05:56:28.885105 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d308561f-4b9e-4c45-8b42-f3ecc3ff7b60 None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:28.885590 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d308561f-4b9e-4c45-8b42-f3ecc3ff7b60 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:28.886032 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d308561f-4b9e-4c45-8b42-f3ecc3ff7b60 None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:28.887448 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 15/32] 10.222.0.62 () {54 vars in 927 bytes} [Wed Aug 24 05:56:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:56:28.888351 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d308561f-4b9e-4c45-8b42-f3ecc3ff7b60 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:56:28.965772 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a6f484-afa1-4943-8e3a-325481685614 None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:56:28.998989 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:56:29.010049 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d308561f-4b9e-4c45-8b42-f3ecc3ff7b60 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:56:29.019962 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f074869-0080-4114-9f15-f49d8bfa5ede None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:56:29.020955 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f074869-0080-4114-9f15-f49d8bfa5ede None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:29.021435 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f074869-0080-4114-9f15-f49d8bfa5ede None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:29.021925 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f074869-0080-4114-9f15-f49d8bfa5ede None admin] PATH_INFO: `/v3/users/8b2c3a376af245f2a0e347e2f08885a5` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:29.023801 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f074869-0080-4114-9f15-f49d8bfa5ede None admin] RBAC: Authorizing `identity:get_user(user_id=8b2c3a376af245f2a0e347e2f08885a5)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:56:29.026087 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f074869-0080-4114-9f15-f49d8bfa5ede None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:56:29.028036 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 18/33] 10.222.0.62 () {56 vars in 1177 bytes} [Wed Aug 24 05:56:28 2022] GET /identity/v3/users/8b2c3a376af245f2a0e347e2f08885a5 => generated 247 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:56:29.029622 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ca98c9-eec0-4ac7-853e-5a6aec8be6d8 None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:29.030132 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ca98c9-eec0-4ac7-853e-5a6aec8be6d8 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:29.030603 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ca98c9-eec0-4ac7-853e-5a6aec8be6d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:29.049495 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-a5ca98c9-eec0-4ac7-853e-5a6aec8be6d8 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:56:29.056460 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-a5ca98c9-eec0-4ac7-853e-5a6aec8be6d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:56:29.074101 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f90472-7873-4183-839a-932088ba1905 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:56:29.074808 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f90472-7873-4183-839a-932088ba1905 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:29.075099 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f90472-7873-4183-839a-932088ba1905 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:29.075967 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f90472-7873-4183-839a-932088ba1905 None admin] PATH_INFO: `/v3/domains` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:29.076081 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f90472-7873-4183-839a-932088ba1905 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:56:29.076349 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f90472-7873-4183-839a-932088ba1905 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:56:29.078006 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f90472-7873-4183-839a-932088ba1905 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:56:29.108107 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 16/34] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Aug 24 05:56:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:56:29.108914 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f90472-7873-4183-839a-932088ba1905 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:56:29.116890 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:56:29.131303 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 19/35] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Aug 24 05:56:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 05:56:29.132407 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ca98c9-eec0-4ac7-853e-5a6aec8be6d8 None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:56:29.254811 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ca98c9-eec0-4ac7-853e-5a6aec8be6d8 None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:56:29.299190 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9588a792-3b43-417b-b787-0c1b56c64835 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:56:29.299821 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9588a792-3b43-417b-b787-0c1b56c64835 None admin] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:29.300107 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9588a792-3b43-417b-b787-0c1b56c64835 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:29.300629 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9588a792-3b43-417b-b787-0c1b56c64835 None admin] PATH_INFO: `/v3/projects` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:29.301581 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9588a792-3b43-417b-b787-0c1b56c64835 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:56:29.303047 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9588a792-3b43-417b-b787-0c1b56c64835 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:56:29.330149 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f90472-7873-4183-839a-932088ba1905 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:56:29.368677 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811d4564-08b9-4740-bb45-368664f75aac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:56:29.369355 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811d4564-08b9-4740-bb45-368664f75aac None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:29.369639 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811d4564-08b9-4740-bb45-368664f75aac None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:29.369910 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811d4564-08b9-4740-bb45-368664f75aac None admin] PATH_INFO: `/v3/domains/default` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:29.372646 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811d4564-08b9-4740-bb45-368664f75aac None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:56:29.375170 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811d4564-08b9-4740-bb45-368664f75aac None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:56:29.375399 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 17/36] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Aug 24 05:56:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:56:29.377223 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327f0898-b4f3-48df-8537-f58cd2d8601e None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:29.377223 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327f0898-b4f3-48df-8537-f58cd2d8601e None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:29.377403 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327f0898-b4f3-48df-8537-f58cd2d8601e None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:29.378443 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 18/37] 10.222.0.62 () {54 vars in 927 bytes} [Wed Aug 24 05:56:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:56:29.383947 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327f0898-b4f3-48df-8537-f58cd2d8601e None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:56:29.413067 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:56:29.499416 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327f0898-b4f3-48df-8537-f58cd2d8601e None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:56:29.530261 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 20/38] 10.222.0.62 () {60 vars in 1141 bytes} [Wed Aug 24 05:56:29 2022] POST /identity/v3/projects => generated 301 bytes in 398 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:56:29.531701 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f319c7d5-86a7-4631-8de1-ec7512d2fb3b None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:29.531990 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f319c7d5-86a7-4631-8de1-ec7512d2fb3b None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:29.532263 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f319c7d5-86a7-4631-8de1-ec7512d2fb3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:29.548462 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66204b6e-2449-4aa8-959c-af3e6078f7aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:56:29.549482 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66204b6e-2449-4aa8-959c-af3e6078f7aa None admin] REQUEST_METHOD: `PUT` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:29.549909 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66204b6e-2449-4aa8-959c-af3e6078f7aa None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:29.550329 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66204b6e-2449-4aa8-959c-af3e6078f7aa None admin] PATH_INFO: `/v3/domains/default/users/8b2c3a376af245f2a0e347e2f08885a5/roles/49e47cd157d743f6905f1db07ec878a7` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:29.554538 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66204b6e-2449-4aa8-959c-af3e6078f7aa None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=8b2c3a376af245f2a0e347e2f08885a5, role_id=49e47cd157d743f6905f1db07ec878a7)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:56:29.556819 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66204b6e-2449-4aa8-959c-af3e6078f7aa None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:56:29.559666 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-f319c7d5-86a7-4631-8de1-ec7512d2fb3b None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:56:29.561544 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-f319c7d5-86a7-4631-8de1-ec7512d2fb3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:56:29.597983 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:56:29.648525 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 21/39] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Aug 24 05:56:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 05:56:29.648525 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f319c7d5-86a7-4631-8de1-ec7512d2fb3b None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:56:29.678992 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 19/40] 10.222.0.62 () {58 vars in 1361 bytes} [Wed Aug 24 05:56:29 2022] PUT /identity/v3/domains/default/users/8b2c3a376af245f2a0e347e2f08885a5/roles/49e47cd157d743f6905f1db07ec878a7 => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:56:29.740707 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d2587d-0164-44ed-9d7b-fe2ebc4756d1 None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:29.741392 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d2587d-0164-44ed-9d7b-fe2ebc4756d1 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:29.741882 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d2587d-0164-44ed-9d7b-fe2ebc4756d1 None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:29.744048 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 20/41] 10.222.0.62 () {54 vars in 927 bytes} [Wed Aug 24 05:56:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:56:29.749875 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c2cafb-cfe2-4b5d-96b2-da733638b52e None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:29.757996 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c2cafb-cfe2-4b5d-96b2-da733638b52e None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:29.757996 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c2cafb-cfe2-4b5d-96b2-da733638b52e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:29.794906 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-37c2cafb-cfe2-4b5d-96b2-da733638b52e None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:56:29.794906 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-37c2cafb-cfe2-4b5d-96b2-da733638b52e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:56:29.900411 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 21/42] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Aug 24 05:56:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 05:56:29.901645 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f319c7d5-86a7-4631-8de1-ec7512d2fb3b None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:56:29.943952 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dedac55-7545-4fbd-a259-53f669a42a8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:56:29.945014 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dedac55-7545-4fbd-a259-53f669a42a8d None admin] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:29.945495 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dedac55-7545-4fbd-a259-53f669a42a8d None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:29.945963 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dedac55-7545-4fbd-a259-53f669a42a8d None admin] PATH_INFO: `/v3/roles` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:29.946822 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dedac55-7545-4fbd-a259-53f669a42a8d None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:56:29.948636 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dedac55-7545-4fbd-a259-53f669a42a8d None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:56:29.972518 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c2cafb-cfe2-4b5d-96b2-da733638b52e None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:56:29.991020 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:56:30.053717 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 22/43] 10.222.0.62 () {60 vars in 1131 bytes} [Wed Aug 24 05:56:29 2022] POST /identity/v3/roles => generated 225 bytes in 413 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:56:30.091720 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c2cafb-cfe2-4b5d-96b2-da733638b52e None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:56:30.131999 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594689c3-59c1-40bf-8441-af8a1b637a96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:56:30.132718 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594689c3-59c1-40bf-8441-af8a1b637a96 None admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:30.133019 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594689c3-59c1-40bf-8441-af8a1b637a96 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:30.133350 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594689c3-59c1-40bf-8441-af8a1b637a96 None admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:30.134043 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594689c3-59c1-40bf-8441-af8a1b637a96 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:56:30.135523 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594689c3-59c1-40bf-8441-af8a1b637a96 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:56:30.192416 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dedac55-7545-4fbd-a259-53f669a42a8d None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:56:30.223790 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:56:30.294003 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 22/44] 10.222.0.62 () {60 vars in 1131 bytes} [Wed Aug 24 05:56:29 2022] POST /identity/v3/roles => generated 223 bytes in 322 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 05:56:30.298395 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dedac55-7545-4fbd-a259-53f669a42a8d None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:56:30.338170 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f335b3c-4494-46b0-8806-e6a9193c8a9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:56:30.338756 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f335b3c-4494-46b0-8806-e6a9193c8a9b None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:30.339089 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f335b3c-4494-46b0-8806-e6a9193c8a9b None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:30.339448 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f335b3c-4494-46b0-8806-e6a9193c8a9b None admin] PATH_INFO: `/v3/domains/default` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:30.340622 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f335b3c-4494-46b0-8806-e6a9193c8a9b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:56:30.342255 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f335b3c-4494-46b0-8806-e6a9193c8a9b None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:56:30.344379 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 23/45] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Aug 24 05:56:30 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:56:30.350084 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594689c3-59c1-40bf-8441-af8a1b637a96 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:56:30.434612 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:56:30.463043 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594689c3-59c1-40bf-8441-af8a1b637a96 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:56:30.497153 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5614558f-0407-4817-8da0-cc49401b2263 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:56:30.497781 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5614558f-0407-4817-8da0-cc49401b2263 None admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:30.498064 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5614558f-0407-4817-8da0-cc49401b2263 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:30.498338 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5614558f-0407-4817-8da0-cc49401b2263 None admin] PATH_INFO: `/v3/projects` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:30.498993 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5614558f-0407-4817-8da0-cc49401b2263 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:56:30.500450 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5614558f-0407-4817-8da0-cc49401b2263 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:56:30.590675 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 23/46] 10.222.0.62 () {60 vars in 1141 bytes} [Wed Aug 24 05:56:30 2022] POST /identity/v3/projects => generated 312 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:56:31.635001 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e97d12-623f-49ca-87f1-9603f85e0944 None None] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:31.635548 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e97d12-623f-49ca-87f1-9603f85e0944 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:31.635548 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e97d12-623f-49ca-87f1-9603f85e0944 None None] PATH_INFO: `/` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:31.637081 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 24/47] 10.222.0.62 () {54 vars in 927 bytes} [Wed Aug 24 05:56:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:56:31.643297 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e595fa-c75f-4d17-a4ed-c0b537223291 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:31.643851 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e595fa-c75f-4d17-a4ed-c0b537223291 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:31.643851 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e595fa-c75f-4d17-a4ed-c0b537223291 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:31.662142 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-c4e595fa-c75f-4d17-a4ed-c0b537223291 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:56:31.663322 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-c4e595fa-c75f-4d17-a4ed-c0b537223291 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:56:31.729110 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 24/48] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Aug 24 05:56:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 05:56:31.998297 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e97d12-623f-49ca-87f1-9603f85e0944 None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:56:32.092286 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:56:32.098869 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e97d12-623f-49ca-87f1-9603f85e0944 None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:56:32.132040 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669ab278-a6de-4c17-a28e-cfb6d41813c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:56:32.132697 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669ab278-a6de-4c17-a28e-cfb6d41813c9 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:32.132983 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669ab278-a6de-4c17-a28e-cfb6d41813c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:32.133261 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669ab278-a6de-4c17-a28e-cfb6d41813c9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:32.143249 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669ab278-a6de-4c17-a28e-cfb6d41813c9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:56:32.148104 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application [None req-669ab278-a6de-4c17-a28e-cfb6d41813c9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 05:56:32.148104 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:56:32.148104 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:56:32.148104 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:56:32.148104 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:56:32.148104 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:56:32.148104 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:56:32.148104 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:56:32.148104 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:56:32.148104 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:56:32.148104 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:56:32.148104 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:56:32.148104 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:56:32.148104 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:56:32.148104 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:56:32.148104 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:56:32.149164 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:56:32.149164 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:56:32.149164 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:56:32.149164 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:56:32.149164 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:56:32.149164 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:56:32.149164 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:56:32.149164 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:56:32.149164 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:56:32.149164 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:56:32.149164 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:56:32.149164 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application with Lock( Aug 24 05:56:32.149164 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:56:32.149164 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:56:32.149164 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:56:32.149164 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:56:32.150098 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:56:32.150098 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:56:32.150098 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:56:32.150098 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:56:32.150098 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:56:32.150098 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:56:32.150098 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:56:32.150098 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:56:32.150098 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:56:32.150098 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:56:32.150098 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 05:56:32.150098 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Aug 24 05:56:32.150098 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 25/49] 10.222.0.62 () {56 vars in 1096 bytes} [Wed Aug 24 05:56:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:56:32.155028 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e595fa-c75f-4d17-a4ed-c0b537223291 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:56:32.212431 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:56:32.255961 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e595fa-c75f-4d17-a4ed-c0b537223291 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:56:32.288066 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf4e9c6-073c-4779-91e7-555e58c9c948 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:56:32.288701 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf4e9c6-073c-4779-91e7-555e58c9c948 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:32.288986 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf4e9c6-073c-4779-91e7-555e58c9c948 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:32.289262 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf4e9c6-073c-4779-91e7-555e58c9c948 None admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:32.289923 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf4e9c6-073c-4779-91e7-555e58c9c948 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:56:32.290229 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf4e9c6-073c-4779-91e7-555e58c9c948 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:56:32.291673 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf4e9c6-073c-4779-91e7-555e58c9c948 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:56:32.304783 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 25/50] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Aug 24 05:56:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:56:32.310328 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669ab278-a6de-4c17-a28e-cfb6d41813c9 None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:56:32.407651 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669ab278-a6de-4c17-a28e-cfb6d41813c9 None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:56:32.442447 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1fba60-5b05-47c9-be10-05284a90dff3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:56:32.443072 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1fba60-5b05-47c9-be10-05284a90dff3 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:32.443356 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1fba60-5b05-47c9-be10-05284a90dff3 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:32.443629 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1fba60-5b05-47c9-be10-05284a90dff3 None admin] PATH_INFO: `/v3/users/8b2c3a376af245f2a0e347e2f08885a5` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:32.444779 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1fba60-5b05-47c9-be10-05284a90dff3 None admin] RBAC: Authorizing `identity:get_user(user_id=8b2c3a376af245f2a0e347e2f08885a5)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:56:32.446231 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1fba60-5b05-47c9-be10-05284a90dff3 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:56:32.448022 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 26/51] 10.222.0.62 () {56 vars in 1177 bytes} [Wed Aug 24 05:56:32 2022] GET /identity/v3/users/8b2c3a376af245f2a0e347e2f08885a5 => generated 247 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:56:32.453418 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf4e9c6-073c-4779-91e7-555e58c9c948 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:56:32.462837 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe73c36-2ef0-4e13-8fe0-dc407f29d9d5 None None] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:32.463123 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe73c36-2ef0-4e13-8fe0-dc407f29d9d5 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:32.463403 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe73c36-2ef0-4e13-8fe0-dc407f29d9d5 None None] PATH_INFO: `/` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:32.464741 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 27/52] 10.222.0.62 () {54 vars in 927 bytes} [Wed Aug 24 05:56:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 05:56:32.469964 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c670fa4-7e70-4925-b6ef-81599ff8dbe0 None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:32.470403 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c670fa4-7e70-4925-b6ef-81599ff8dbe0 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:32.470676 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c670fa4-7e70-4925-b6ef-81599ff8dbe0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:32.489376 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-4c670fa4-7e70-4925-b6ef-81599ff8dbe0 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:56:32.490593 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-4c670fa4-7e70-4925-b6ef-81599ff8dbe0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:56:32.508188 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:56:32.557949 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf4e9c6-073c-4779-91e7-555e58c9c948 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:56:32.558801 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 28/53] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Aug 24 05:56:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 05:56:32.576401 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:56:32.592921 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1eb2db-673b-4ca4-9d6a-72b27e2b5306 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:56:32.593545 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1eb2db-673b-4ca4-9d6a-72b27e2b5306 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:32.593824 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1eb2db-673b-4ca4-9d6a-72b27e2b5306 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:32.594095 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1eb2db-673b-4ca4-9d6a-72b27e2b5306 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:32.595269 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1eb2db-673b-4ca4-9d6a-72b27e2b5306 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:56:32.596744 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1eb2db-673b-4ca4-9d6a-72b27e2b5306 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:56:32.598855 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 26/54] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Aug 24 05:56:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:56:32.604764 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c670fa4-7e70-4925-b6ef-81599ff8dbe0 None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:56:32.710413 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c670fa4-7e70-4925-b6ef-81599ff8dbe0 None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:56:32.743306 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f75e22-bc32-4bd5-b862-8d7b176f6a83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:56:32.743895 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f75e22-bc32-4bd5-b862-8d7b176f6a83 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:32.744174 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f75e22-bc32-4bd5-b862-8d7b176f6a83 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:32.744442 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f75e22-bc32-4bd5-b862-8d7b176f6a83 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:32.745180 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f75e22-bc32-4bd5-b862-8d7b176f6a83 None admin] RBAC: Adding query filter params (role.id=49e47cd157d743f6905f1db07ec878a7, scope.domain.id=default, user.id=8b2c3a376af245f2a0e347e2f08885a5) {{(pid=81851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:56:32.745487 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f75e22-bc32-4bd5-b862-8d7b176f6a83 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:56:32.746910 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f75e22-bc32-4bd5-b862-8d7b176f6a83 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:56:32.752660 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 29/55] 10.222.0.62 () {56 vars in 1322 bytes} [Wed Aug 24 05:56:32 2022] GET /identity/v3/role_assignments?user.id=8b2c3a376af245f2a0e347e2f08885a5&scope.domain.id=default&role.id=49e47cd157d743f6905f1db07ec878a7 => generated 530 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:56:32.834337 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1eb2db-673b-4ca4-9d6a-72b27e2b5306 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:56:33.038253 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1eb2db-673b-4ca4-9d6a-72b27e2b5306 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:56:33.070356 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256cebd5-203e-43f5-9549-ee06521f8245 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:56:33.070970 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256cebd5-203e-43f5-9549-ee06521f8245 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:33.071247 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256cebd5-203e-43f5-9549-ee06521f8245 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:33.071518 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256cebd5-203e-43f5-9549-ee06521f8245 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:33.072614 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256cebd5-203e-43f5-9549-ee06521f8245 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:56:33.074042 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256cebd5-203e-43f5-9549-ee06521f8245 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:56:33.075570 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 27/56] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Aug 24 05:56:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:56:33.081169 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f75e22-bc32-4bd5-b862-8d7b176f6a83 None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:56:33.100721 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:56:33.177957 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f75e22-bc32-4bd5-b862-8d7b176f6a83 None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:56:33.210668 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c170fbb-2c55-4190-97f3-c49cafe8c8d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:56:33.211259 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c170fbb-2c55-4190-97f3-c49cafe8c8d8 None admin] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:33.211553 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c170fbb-2c55-4190-97f3-c49cafe8c8d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:33.211806 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c170fbb-2c55-4190-97f3-c49cafe8c8d8 None admin] PATH_INFO: `/v3/projects` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:33.212436 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c170fbb-2c55-4190-97f3-c49cafe8c8d8 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:56:33.213877 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c170fbb-2c55-4190-97f3-c49cafe8c8d8 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:56:33.299112 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 30/57] 10.222.0.62 () {60 vars in 1140 bytes} [Wed Aug 24 05:56:33 2022] POST /identity/v3/projects => generated 298 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:56:35.181768 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874c0efd-0bcc-401a-8d5b-c12fde4b4b89 None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:35.182276 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874c0efd-0bcc-401a-8d5b-c12fde4b4b89 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:35.182386 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874c0efd-0bcc-401a-8d5b-c12fde4b4b89 None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:35.183650 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 28/58] 10.222.0.62 () {54 vars in 927 bytes} [Wed Aug 24 05:56:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:56:35.188858 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a36b7e-11ef-4203-aafd-5cf3dd5dc9c9 None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:35.189240 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a36b7e-11ef-4203-aafd-5cf3dd5dc9c9 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:35.189416 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a36b7e-11ef-4203-aafd-5cf3dd5dc9c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:35.207420 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-53a36b7e-11ef-4203-aafd-5cf3dd5dc9c9 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:56:35.208756 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-53a36b7e-11ef-4203-aafd-5cf3dd5dc9c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:56:35.269567 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:56:35.273767 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 31/59] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Aug 24 05:56:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 05:56:35.541812 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874c0efd-0bcc-401a-8d5b-c12fde4b4b89 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:56:35.597132 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:56:35.639813 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874c0efd-0bcc-401a-8d5b-c12fde4b4b89 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:56:35.672118 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cbce58-b9a8-47d3-a93f-3c873aea2f69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:56:35.672767 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cbce58-b9a8-47d3-a93f-3c873aea2f69 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:35.673045 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cbce58-b9a8-47d3-a93f-3c873aea2f69 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:35.673315 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cbce58-b9a8-47d3-a93f-3c873aea2f69 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:35.674398 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1cbce58-b9a8-47d3-a93f-3c873aea2f69 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:56:35.675869 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1cbce58-b9a8-47d3-a93f-3c873aea2f69 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:56:35.677431 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 29/60] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Aug 24 05:56:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:56:35.682966 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a36b7e-11ef-4203-aafd-5cf3dd5dc9c9 None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:56:35.784141 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a36b7e-11ef-4203-aafd-5cf3dd5dc9c9 None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:56:35.820097 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffac1057-2a38-49b5-8d2f-d4e17ef3740d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:56:35.820673 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffac1057-2a38-49b5-8d2f-d4e17ef3740d None admin] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:35.820959 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffac1057-2a38-49b5-8d2f-d4e17ef3740d None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:35.821235 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffac1057-2a38-49b5-8d2f-d4e17ef3740d None admin] PATH_INFO: `/v3/users` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:35.822036 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffac1057-2a38-49b5-8d2f-d4e17ef3740d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:56:35.823498 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffac1057-2a38-49b5-8d2f-d4e17ef3740d None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:56:35.925578 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 32/61] 10.222.0.62 () {60 vars in 1132 bytes} [Wed Aug 24 05:56:35 2022] POST /identity/v3/users => generated 275 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:56:38.836796 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de5788b-1250-433c-ae50-882564ca74f0 None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:38.837307 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de5788b-1250-433c-ae50-882564ca74f0 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:38.837378 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de5788b-1250-433c-ae50-882564ca74f0 None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:38.838792 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 30/62] 10.222.0.62 () {54 vars in 927 bytes} [Wed Aug 24 05:56:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:56:38.844232 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6058fb7-9643-4e84-b667-024d9b62ebf3 None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:38.844705 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6058fb7-9643-4e84-b667-024d9b62ebf3 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:38.844873 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6058fb7-9643-4e84-b667-024d9b62ebf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:38.869434 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-f6058fb7-9643-4e84-b667-024d9b62ebf3 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:56:38.870617 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-f6058fb7-9643-4e84-b667-024d9b62ebf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:56:38.935910 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:56:38.958344 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 33/63] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Aug 24 05:56:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 05:56:38.962337 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d481c0-6615-4da8-a7bd-eee89104d8ae None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:38.962625 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d481c0-6615-4da8-a7bd-eee89104d8ae None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:38.962901 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d481c0-6615-4da8-a7bd-eee89104d8ae None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:38.964413 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 31/64] 10.222.0.62 () {54 vars in 927 bytes} [Wed Aug 24 05:56:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:56:38.969788 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4179f533-7469-479e-9b9c-d03353def6a7 None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:38.970076 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4179f533-7469-479e-9b9c-d03353def6a7 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:38.970354 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4179f533-7469-479e-9b9c-d03353def6a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:38.988094 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-4179f533-7469-479e-9b9c-d03353def6a7 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:56:38.989301 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-4179f533-7469-479e-9b9c-d03353def6a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:56:39.076977 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 34/65] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Aug 24 05:56:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 05:56:39.182533 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7264a9ae-fa59-4816-a152-e5eba539e984 None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:39.182867 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7264a9ae-fa59-4816-a152-e5eba539e984 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:39.183094 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7264a9ae-fa59-4816-a152-e5eba539e984 None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:39.184430 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 32/66] 10.222.0.62 () {54 vars in 927 bytes} [Wed Aug 24 05:56:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:56:39.189745 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d72152-0224-4885-8268-e0c33801c8b8 None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:39.190037 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d72152-0224-4885-8268-e0c33801c8b8 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:39.190318 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d72152-0224-4885-8268-e0c33801c8b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:39.229545 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-76d72152-0224-4885-8268-e0c33801c8b8 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:56:39.230643 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-76d72152-0224-4885-8268-e0c33801c8b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:56:39.308114 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7264a9ae-fa59-4816-a152-e5eba539e984 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:56:39.324223 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 35/67] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Aug 24 05:56:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 05:56:39.434808 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d72152-0224-4885-8268-e0c33801c8b8 None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:56:39.475826 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7264a9ae-fa59-4816-a152-e5eba539e984 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:56:39.526977 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:56:39.544618 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:56:39.551314 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc5d37d-f96f-402b-b4ff-eb5ddffdcfbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:56:39.551959 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc5d37d-f96f-402b-b4ff-eb5ddffdcfbc None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:39.552241 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc5d37d-f96f-402b-b4ff-eb5ddffdcfbc None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:39.552537 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc5d37d-f96f-402b-b4ff-eb5ddffdcfbc None admin] PATH_INFO: `/v3/roles/member` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:39.574650 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc5d37d-f96f-402b-b4ff-eb5ddffdcfbc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:56:39.579706 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-8bc5d37d-f96f-402b-b4ff-eb5ddffdcfbc None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 05:56:39.579706 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:56:39.579706 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:56:39.579706 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:56:39.579706 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:56:39.579706 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:56:39.579706 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:56:39.579706 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:56:39.579706 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:56:39.579706 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:56:39.579706 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:56:39.579706 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:56:39.579706 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:56:39.579706 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:56:39.579706 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:56:39.579706 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:56:39.580564 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:56:39.580564 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:56:39.580564 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:56:39.580564 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:56:39.580564 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:56:39.580564 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:56:39.580564 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:56:39.580564 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:56:39.580564 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:56:39.580564 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:56:39.580564 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:56:39.580564 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 24 05:56:39.580564 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:56:39.580564 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:56:39.580564 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:56:39.580564 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:56:39.581367 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:56:39.581367 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:56:39.581367 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:56:39.581367 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:56:39.581367 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:56:39.581367 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:56:39.581367 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:56:39.581367 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:56:39.581367 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:56:39.581367 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:56:39.581367 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 05:56:39.581367 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 24 05:56:39.581367 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 33/68] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Aug 24 05:56:39 2022] GET /identity/v3/roles/member => generated 84 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:56:39.587822 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc5d37d-f96f-402b-b4ff-eb5ddffdcfbc None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:56:39.652137 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d72152-0224-4885-8268-e0c33801c8b8 None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:56:39.699796 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238f6c1d-17d4-4778-9b24-54ab6e59637a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:56:39.700439 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238f6c1d-17d4-4778-9b24-54ab6e59637a None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:39.700760 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238f6c1d-17d4-4778-9b24-54ab6e59637a None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:39.701044 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238f6c1d-17d4-4778-9b24-54ab6e59637a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:39.715989 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238f6c1d-17d4-4778-9b24-54ab6e59637a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:56:39.717706 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc5d37d-f96f-402b-b4ff-eb5ddffdcfbc None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:56:39.720930 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application [None req-238f6c1d-17d4-4778-9b24-54ab6e59637a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 05:56:39.720930 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:56:39.720930 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:56:39.720930 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:56:39.720930 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:56:39.720930 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:56:39.720930 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:56:39.720930 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:56:39.720930 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:56:39.720930 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:56:39.720930 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:56:39.720930 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:56:39.720930 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:56:39.720930 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:56:39.720930 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:56:39.720930 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:56:39.721774 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:56:39.721774 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:56:39.721774 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:56:39.721774 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:56:39.721774 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:56:39.721774 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:56:39.721774 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:56:39.721774 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:56:39.721774 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:56:39.721774 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:56:39.721774 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:56:39.721774 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application with Lock( Aug 24 05:56:39.721774 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:56:39.721774 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:56:39.721774 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:56:39.721774 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:56:39.723744 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:56:39.723744 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:56:39.723744 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:56:39.723744 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:56:39.723744 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:56:39.723744 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:56:39.723744 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:56:39.723744 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:56:39.723744 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:56:39.723744 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:56:39.723744 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 05:56:39.723744 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Aug 24 05:56:39.723744 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 36/69] 10.222.0.62 () {56 vars in 1096 bytes} [Wed Aug 24 05:56:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 05:56:39.724382 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238f6c1d-17d4-4778-9b24-54ab6e59637a None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:56:39.760467 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32caabc-5071-440b-9445-cf7724912976 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:56:39.761119 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32caabc-5071-440b-9445-cf7724912976 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:39.761399 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32caabc-5071-440b-9445-cf7724912976 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:39.761681 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32caabc-5071-440b-9445-cf7724912976 None admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:39.762327 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32caabc-5071-440b-9445-cf7724912976 None admin] RBAC: Adding query filter params (name=member) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:56:39.763311 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32caabc-5071-440b-9445-cf7724912976 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:56:39.764044 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32caabc-5071-440b-9445-cf7724912976 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:56:39.773224 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 34/70] 10.222.0.62 () {56 vars in 1101 bytes} [Wed Aug 24 05:56:39 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:56:39.777933 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8adda5-43b2-4e6a-9d5f-6623dbcb194d None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:39.778126 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8adda5-43b2-4e6a-9d5f-6623dbcb194d None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:39.778336 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8adda5-43b2-4e6a-9d5f-6623dbcb194d None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:39.779585 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 35/71] 10.222.0.62 () {54 vars in 927 bytes} [Wed Aug 24 05:56:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:56:39.780759 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8adda5-43b2-4e6a-9d5f-6623dbcb194d None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:56:39.831129 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:56:39.840724 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238f6c1d-17d4-4778-9b24-54ab6e59637a None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:56:39.849479 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:56:39.878813 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25639272-18d2-431b-8629-6212713f433f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:56:39.879463 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25639272-18d2-431b-8629-6212713f433f None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:39.879822 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25639272-18d2-431b-8629-6212713f433f None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:39.880030 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25639272-18d2-431b-8629-6212713f433f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:39.890870 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25639272-18d2-431b-8629-6212713f433f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:56:39.893447 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8adda5-43b2-4e6a-9d5f-6623dbcb194d None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:56:39.895961 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application [None req-25639272-18d2-431b-8629-6212713f433f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 05:56:39.895961 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:56:39.895961 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:56:39.895961 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:56:39.895961 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:56:39.895961 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:56:39.895961 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:56:39.895961 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:56:39.895961 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:56:39.895961 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:56:39.895961 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:56:39.895961 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:56:39.895961 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:56:39.895961 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:56:39.895961 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:56:39.895961 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:56:39.896859 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:56:39.896859 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:56:39.896859 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:56:39.896859 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:56:39.896859 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:56:39.896859 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:56:39.896859 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:56:39.896859 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:56:39.896859 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:56:39.896859 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:56:39.896859 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:56:39.896859 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application with Lock( Aug 24 05:56:39.896859 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:56:39.896859 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:56:39.896859 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:56:39.896859 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:56:39.897638 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:56:39.897638 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:56:39.897638 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:56:39.897638 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:56:39.897638 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:56:39.897638 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:56:39.897638 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:56:39.897638 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:56:39.897638 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:56:39.897638 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:56:39.897638 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 05:56:39.897638 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Aug 24 05:56:39.902344 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 37/72] 10.222.0.62 () {56 vars in 1114 bytes} [Wed Aug 24 05:56:39 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:56:39.903052 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25639272-18d2-431b-8629-6212713f433f None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:56:39.936121 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d054af39-68a7-4795-9800-712ff4fc84a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:56:39.936746 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d054af39-68a7-4795-9800-712ff4fc84a6 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:39.937028 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d054af39-68a7-4795-9800-712ff4fc84a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:39.937296 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d054af39-68a7-4795-9800-712ff4fc84a6 None admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:39.937970 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d054af39-68a7-4795-9800-712ff4fc84a6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:56:39.938252 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d054af39-68a7-4795-9800-712ff4fc84a6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:56:39.939638 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d054af39-68a7-4795-9800-712ff4fc84a6 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:56:39.948548 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 36/73] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Aug 24 05:56:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:56:39.949719 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6834523-9537-48b1-a437-1db17b8544b4 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:39.950008 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6834523-9537-48b1-a437-1db17b8544b4 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:39.950282 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6834523-9537-48b1-a437-1db17b8544b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:39.968661 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-f6834523-9537-48b1-a437-1db17b8544b4 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:56:39.969977 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-f6834523-9537-48b1-a437-1db17b8544b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:56:40.013954 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25639272-18d2-431b-8629-6212713f433f None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:56:40.037798 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 37/74] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Aug 24 05:56:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 05:56:40.038556 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6834523-9537-48b1-a437-1db17b8544b4 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:56:40.049844 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709208f2-f66e-417e-9292-379dc76c6549 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:56:40.051388 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709208f2-f66e-417e-9292-379dc76c6549 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:40.051676 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709208f2-f66e-417e-9292-379dc76c6549 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:40.051948 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709208f2-f66e-417e-9292-379dc76c6549 None admin] PATH_INFO: `/v3/users/f16ca5f6955045a5811c25a8e6cc8055` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:40.067010 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709208f2-f66e-417e-9292-379dc76c6549 None admin] RBAC: Authorizing `identity:get_user(user_id=f16ca5f6955045a5811c25a8e6cc8055)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:56:40.068526 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709208f2-f66e-417e-9292-379dc76c6549 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:56:40.070296 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 38/75] 10.222.0.62 () {56 vars in 1177 bytes} [Wed Aug 24 05:56:39 2022] GET /identity/v3/users/f16ca5f6955045a5811c25a8e6cc8055 => generated 275 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:56:40.070896 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709208f2-f66e-417e-9292-379dc76c6549 None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:56:40.078852 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:56:40.096710 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:56:40.144777 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6834523-9537-48b1-a437-1db17b8544b4 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:56:40.176872 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709208f2-f66e-417e-9292-379dc76c6549 None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:56:40.179802 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a75aa6-99de-419d-96d6-161caa89ba8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:56:40.180107 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a75aa6-99de-419d-96d6-161caa89ba8d None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:40.180411 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a75aa6-99de-419d-96d6-161caa89ba8d None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:40.180716 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a75aa6-99de-419d-96d6-161caa89ba8d None admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:40.181376 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a75aa6-99de-419d-96d6-161caa89ba8d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:56:40.181680 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a75aa6-99de-419d-96d6-161caa89ba8d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:56:40.183054 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a75aa6-99de-419d-96d6-161caa89ba8d None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:56:40.191294 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 38/76] 10.222.0.62 () {56 vars in 1111 bytes} [Wed Aug 24 05:56:40 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:56:40.192281 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae0534b-0fda-4427-8f05-18757cd9f4e0 None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:40.192600 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae0534b-0fda-4427-8f05-18757cd9f4e0 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:40.192881 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae0534b-0fda-4427-8f05-18757cd9f4e0 None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:40.194079 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 39/77] 10.222.0.62 () {54 vars in 927 bytes} [Wed Aug 24 05:56:40 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:56:40.197566 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae0534b-0fda-4427-8f05-18757cd9f4e0 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:56:40.217780 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f9a24e-2bf0-40b6-bf9a-626c6f4a7363 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:56:40.218414 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f9a24e-2bf0-40b6-bf9a-626c6f4a7363 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:40.218693 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f9a24e-2bf0-40b6-bf9a-626c6f4a7363 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:40.218966 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f9a24e-2bf0-40b6-bf9a-626c6f4a7363 None admin] PATH_INFO: `/v3/users/8b2c3a376af245f2a0e347e2f08885a5` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:40.220078 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f9a24e-2bf0-40b6-bf9a-626c6f4a7363 None admin] RBAC: Authorizing `identity:get_user(user_id=8b2c3a376af245f2a0e347e2f08885a5)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:56:40.221699 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f9a24e-2bf0-40b6-bf9a-626c6f4a7363 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:56:40.228845 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 39/78] 10.222.0.62 () {56 vars in 1177 bytes} [Wed Aug 24 05:56:40 2022] GET /identity/v3/users/8b2c3a376af245f2a0e347e2f08885a5 => generated 247 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:56:40.228845 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f9a24e-2bf0-40b6-bf9a-626c6f4a7363 None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:56:40.306220 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:56:40.310771 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae0534b-0fda-4427-8f05-18757cd9f4e0 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:56:40.317800 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:56:40.326196 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f9a24e-2bf0-40b6-bf9a-626c6f4a7363 None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:56:40.343851 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4808f99e-2b97-4e64-9ecc-2a167d35824c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:56:40.344452 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4808f99e-2b97-4e64-9ecc-2a167d35824c None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:40.344760 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4808f99e-2b97-4e64-9ecc-2a167d35824c None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:40.345035 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4808f99e-2b97-4e64-9ecc-2a167d35824c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:40.360116 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1e4b07-f12f-479d-91ec-0c61638d96a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:56:40.360765 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1e4b07-f12f-479d-91ec-0c61638d96a9 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:40.361044 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1e4b07-f12f-479d-91ec-0c61638d96a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:40.361317 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1e4b07-f12f-479d-91ec-0c61638d96a9 None admin] PATH_INFO: `/v3/users/f16ca5f6955045a5811c25a8e6cc8055` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:40.362450 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1e4b07-f12f-479d-91ec-0c61638d96a9 None admin] RBAC: Authorizing `identity:get_user(user_id=f16ca5f6955045a5811c25a8e6cc8055)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:56:40.363880 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1e4b07-f12f-479d-91ec-0c61638d96a9 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:56:40.365481 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 40/79] 10.222.0.62 () {56 vars in 1177 bytes} [Wed Aug 24 05:56:40 2022] GET /identity/v3/users/f16ca5f6955045a5811c25a8e6cc8055 => generated 275 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:56:40.366748 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad79567-adfe-4bbf-9b8c-46c8f3cd1987 None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:40.367029 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad79567-adfe-4bbf-9b8c-46c8f3cd1987 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:40.367301 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad79567-adfe-4bbf-9b8c-46c8f3cd1987 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:40.385088 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-9ad79567-adfe-4bbf-9b8c-46c8f3cd1987 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:56:40.386268 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-9ad79567-adfe-4bbf-9b8c-46c8f3cd1987 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:56:40.417322 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4808f99e-2b97-4e64-9ecc-2a167d35824c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:56:40.418784 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4808f99e-2b97-4e64-9ecc-2a167d35824c None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:56:40.451191 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 41/80] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Aug 24 05:56:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 05:56:40.452089 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad79567-adfe-4bbf-9b8c-46c8f3cd1987 None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:56:40.526566 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:56:40.530872 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 40/81] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Aug 24 05:56:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 05:56:40.531576 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4808f99e-2b97-4e64-9ecc-2a167d35824c None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:56:40.551078 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:56:40.561200 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad79567-adfe-4bbf-9b8c-46c8f3cd1987 None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:56:40.597014 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03db4659-6e60-4b34-b16a-d05469e186ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:56:40.597638 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03db4659-6e60-4b34-b16a-d05469e186ba None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:40.597919 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03db4659-6e60-4b34-b16a-d05469e186ba None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:40.598194 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03db4659-6e60-4b34-b16a-d05469e186ba None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:40.639258 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4808f99e-2b97-4e64-9ecc-2a167d35824c None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:56:40.668647 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03db4659-6e60-4b34-b16a-d05469e186ba None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:56:40.670132 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03db4659-6e60-4b34-b16a-d05469e186ba None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:56:40.685850 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5ded83-b0e2-40f3-8156-558ffb483715 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:56:40.686989 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5ded83-b0e2-40f3-8156-558ffb483715 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:40.687675 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5ded83-b0e2-40f3-8156-558ffb483715 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:40.688092 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5ded83-b0e2-40f3-8156-558ffb483715 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:40.702577 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5ded83-b0e2-40f3-8156-558ffb483715 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:56:40.709010 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-3e5ded83-b0e2-40f3-8156-558ffb483715 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 05:56:40.709010 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:56:40.709010 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:56:40.709010 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:56:40.709010 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:56:40.709010 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:56:40.709010 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:56:40.709010 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:56:40.709010 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:56:40.709010 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:56:40.709010 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:56:40.709010 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:56:40.709010 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:56:40.709010 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:56:40.709010 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:56:40.709010 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:56:40.709916 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:56:40.709916 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:56:40.709916 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:56:40.709916 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:56:40.709916 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:56:40.709916 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:56:40.709916 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:56:40.709916 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:56:40.709916 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:56:40.709916 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:56:40.709916 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:56:40.709916 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 24 05:56:40.709916 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:56:40.709916 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:56:40.709916 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:56:40.709916 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:56:40.710682 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:56:40.710682 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:56:40.710682 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:56:40.710682 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:56:40.710682 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:56:40.710682 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:56:40.710682 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:56:40.710682 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:56:40.710682 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:56:40.710682 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:56:40.710682 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 05:56:40.710682 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 24 05:56:40.710682 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 41/82] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Aug 24 05:56:40 2022] GET /identity/v3/roles/member => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:56:40.712128 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5ded83-b0e2-40f3-8156-558ffb483715 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:56:40.759903 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:56:40.770521 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 42/83] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Aug 24 05:56:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 05:56:40.780893 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03db4659-6e60-4b34-b16a-d05469e186ba None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:56:40.781057 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:56:40.814712 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5ded83-b0e2-40f3-8156-558ffb483715 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:56:40.848088 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e3cd64-121c-4c01-8ff1-505a5da68d0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:56:40.848724 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e3cd64-121c-4c01-8ff1-505a5da68d0c None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:40.849001 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e3cd64-121c-4c01-8ff1-505a5da68d0c None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:40.849272 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e3cd64-121c-4c01-8ff1-505a5da68d0c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:40.868140 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03db4659-6e60-4b34-b16a-d05469e186ba None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:56:40.902089 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec261ab5-1205-4365-a6fd-af457ce68a22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:56:40.902713 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec261ab5-1205-4365-a6fd-af457ce68a22 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:40.902991 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec261ab5-1205-4365-a6fd-af457ce68a22 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:40.903261 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec261ab5-1205-4365-a6fd-af457ce68a22 None admin] PATH_INFO: `/v3/projects/6d7fd5ca249a4df5aba301beb23e2eb7` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:40.904365 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec261ab5-1205-4365-a6fd-af457ce68a22 None admin] RBAC: Authorizing `identity:get_project(project_id=6d7fd5ca249a4df5aba301beb23e2eb7)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:56:40.905831 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec261ab5-1205-4365-a6fd-af457ce68a22 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:56:40.907435 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 43/84] 10.222.0.62 () {56 vars in 1186 bytes} [Wed Aug 24 05:56:40 2022] GET /identity/v3/projects/6d7fd5ca249a4df5aba301beb23e2eb7 => generated 298 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:56:40.908186 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec261ab5-1205-4365-a6fd-af457ce68a22 None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:56:40.915200 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e3cd64-121c-4c01-8ff1-505a5da68d0c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:56:40.916696 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e3cd64-121c-4c01-8ff1-505a5da68d0c None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:56:40.968888 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:56:40.972551 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:56:41.009564 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec261ab5-1205-4365-a6fd-af457ce68a22 None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:56:41.015462 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 42/85] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Aug 24 05:56:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 05:56:41.016293 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e3cd64-121c-4c01-8ff1-505a5da68d0c None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:56:41.043324 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4903d2c8-e885-4b0d-80d7-1cf37045b2c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:56:41.043940 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4903d2c8-e885-4b0d-80d7-1cf37045b2c0 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:41.044224 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4903d2c8-e885-4b0d-80d7-1cf37045b2c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:41.044514 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4903d2c8-e885-4b0d-80d7-1cf37045b2c0 None admin] PATH_INFO: `/v3/roles` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:41.045201 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4903d2c8-e885-4b0d-80d7-1cf37045b2c0 None admin] RBAC: Adding query filter params (name=member) {{(pid=81851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:56:41.045780 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4903d2c8-e885-4b0d-80d7-1cf37045b2c0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:56:41.047156 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4903d2c8-e885-4b0d-80d7-1cf37045b2c0 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:56:41.055825 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 44/86] 10.222.0.62 () {56 vars in 1101 bytes} [Wed Aug 24 05:56:40 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:56:41.056464 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4903d2c8-e885-4b0d-80d7-1cf37045b2c0 None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:56:41.117797 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e3cd64-121c-4c01-8ff1-505a5da68d0c None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:56:41.150526 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436917db-8166-4326-a677-529aa6b580b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:56:41.151152 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436917db-8166-4326-a677-529aa6b580b8 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:41.151429 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436917db-8166-4326-a677-529aa6b580b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:41.151700 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436917db-8166-4326-a677-529aa6b580b8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:41.152439 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4903d2c8-e885-4b0d-80d7-1cf37045b2c0 None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:56:41.152812 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436917db-8166-4326-a677-529aa6b580b8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:56:41.154334 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436917db-8166-4326-a677-529aa6b580b8 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:56:41.155880 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 43/87] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Aug 24 05:56:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:56:41.156716 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436917db-8166-4326-a677-529aa6b580b8 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:56:41.172401 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:56:41.186617 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:56:41.186749 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d94e72-f5d8-4aed-9b48-f13bfe3e8ff9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:56:41.187244 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d94e72-f5d8-4aed-9b48-f13bfe3e8ff9 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:41.187531 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d94e72-f5d8-4aed-9b48-f13bfe3e8ff9 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:41.187813 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d94e72-f5d8-4aed-9b48-f13bfe3e8ff9 None admin] PATH_INFO: `/v3/projects/6d7fd5ca249a4df5aba301beb23e2eb7` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:41.188963 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d94e72-f5d8-4aed-9b48-f13bfe3e8ff9 None admin] RBAC: Authorizing `identity:get_project(project_id=6d7fd5ca249a4df5aba301beb23e2eb7)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:56:41.190430 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d94e72-f5d8-4aed-9b48-f13bfe3e8ff9 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:56:41.192051 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 45/88] 10.222.0.62 () {56 vars in 1186 bytes} [Wed Aug 24 05:56:41 2022] GET /identity/v3/projects/6d7fd5ca249a4df5aba301beb23e2eb7 => generated 298 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:56:41.193138 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d94e72-f5d8-4aed-9b48-f13bfe3e8ff9 None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:56:41.256973 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436917db-8166-4326-a677-529aa6b580b8 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:56:41.290191 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6a7ffe-ebe9-471e-bdac-2317fa378e9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:56:41.290805 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6a7ffe-ebe9-471e-bdac-2317fa378e9f None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:41.291060 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6a7ffe-ebe9-471e-bdac-2317fa378e9f None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:41.291331 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6a7ffe-ebe9-471e-bdac-2317fa378e9f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:41.292040 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6a7ffe-ebe9-471e-bdac-2317fa378e9f None admin] RBAC: Adding query filter params (role.id=580aa7dbb38d4cff8f9d1bb8983f99ef, scope.project.id=6d7fd5ca249a4df5aba301beb23e2eb7, user.id=f16ca5f6955045a5811c25a8e6cc8055) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:56:41.292341 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6a7ffe-ebe9-471e-bdac-2317fa378e9f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:56:41.293804 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6a7ffe-ebe9-471e-bdac-2317fa378e9f None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:56:41.294668 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d94e72-f5d8-4aed-9b48-f13bfe3e8ff9 None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:56:41.302110 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 44/89] 10.222.0.62 () {56 vars in 1374 bytes} [Wed Aug 24 05:56:41 2022] GET /identity/v3/role_assignments?user.id=f16ca5f6955045a5811c25a8e6cc8055&scope.project.id=6d7fd5ca249a4df5aba301beb23e2eb7&role.id=580aa7dbb38d4cff8f9d1bb8983f99ef => generated 259 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:56:41.302710 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6a7ffe-ebe9-471e-bdac-2317fa378e9f None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:56:41.344311 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c50a711-6cf9-427a-a6c8-a171aeaa2770 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:56:41.345586 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c50a711-6cf9-427a-a6c8-a171aeaa2770 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:41.345870 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c50a711-6cf9-427a-a6c8-a171aeaa2770 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:41.346148 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c50a711-6cf9-427a-a6c8-a171aeaa2770 None admin] PATH_INFO: `/v3/projects/6d7fd5ca249a4df5aba301beb23e2eb7` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:41.348383 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c50a711-6cf9-427a-a6c8-a171aeaa2770 None admin] RBAC: Authorizing `identity:get_project(project_id=6d7fd5ca249a4df5aba301beb23e2eb7)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:56:41.350302 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c50a711-6cf9-427a-a6c8-a171aeaa2770 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:56:41.356961 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 46/90] 10.222.0.62 () {56 vars in 1186 bytes} [Wed Aug 24 05:56:41 2022] GET /identity/v3/projects/6d7fd5ca249a4df5aba301beb23e2eb7 => generated 298 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:56:41.356961 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c50a711-6cf9-427a-a6c8-a171aeaa2770 None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:56:41.407904 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:56:41.421646 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6a7ffe-ebe9-471e-bdac-2317fa378e9f None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:56:41.424825 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:56:41.463880 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75b24db-fa87-45c3-aa68-8628243783d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:56:41.464679 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75b24db-fa87-45c3-aa68-8628243783d4 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:41.464972 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75b24db-fa87-45c3-aa68-8628243783d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:41.465245 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75b24db-fa87-45c3-aa68-8628243783d4 None admin] PATH_INFO: `/v3/users/f16ca5f6955045a5811c25a8e6cc8055` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:41.466400 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75b24db-fa87-45c3-aa68-8628243783d4 None admin] RBAC: Authorizing `identity:get_user(user_id=f16ca5f6955045a5811c25a8e6cc8055)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:56:41.467915 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75b24db-fa87-45c3-aa68-8628243783d4 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:56:41.469566 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 45/91] 10.222.0.62 () {56 vars in 1177 bytes} [Wed Aug 24 05:56:41 2022] GET /identity/v3/users/f16ca5f6955045a5811c25a8e6cc8055 => generated 275 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:56:41.469566 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75b24db-fa87-45c3-aa68-8628243783d4 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:56:41.476227 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c50a711-6cf9-427a-a6c8-a171aeaa2770 None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:56:41.513582 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e1b3dd-c815-448b-8831-9da8f1936072 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:56:41.514685 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e1b3dd-c815-448b-8831-9da8f1936072 None admin] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:41.514968 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e1b3dd-c815-448b-8831-9da8f1936072 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:41.515238 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e1b3dd-c815-448b-8831-9da8f1936072 None admin] PATH_INFO: `/v3/users` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:41.515881 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e1b3dd-c815-448b-8831-9da8f1936072 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:56:41.517367 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e1b3dd-c815-448b-8831-9da8f1936072 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:56:41.577173 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75b24db-fa87-45c3-aa68-8628243783d4 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:56:41.596528 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 47/92] 10.222.0.62 () {60 vars in 1132 bytes} [Wed Aug 24 05:56:41 2022] POST /identity/v3/users => generated 289 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:56:41.596528 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e1b3dd-c815-448b-8831-9da8f1936072 None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:56:41.631429 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ddd45f-afdb-4b87-a3fc-4f5939a1723f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:56:41.632147 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ddd45f-afdb-4b87-a3fc-4f5939a1723f None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:41.632514 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ddd45f-afdb-4b87-a3fc-4f5939a1723f None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:41.632860 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ddd45f-afdb-4b87-a3fc-4f5939a1723f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:41.633783 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ddd45f-afdb-4b87-a3fc-4f5939a1723f None admin] RBAC: Adding query filter params (role.id=49e47cd157d743f6905f1db07ec878a7, scope.project.id=6d7fd5ca249a4df5aba301beb23e2eb7, user.id=8b2c3a376af245f2a0e347e2f08885a5) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:56:41.634286 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ddd45f-afdb-4b87-a3fc-4f5939a1723f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:56:41.635770 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ddd45f-afdb-4b87-a3fc-4f5939a1723f None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:56:41.642346 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 46/93] 10.222.0.62 () {56 vars in 1374 bytes} [Wed Aug 24 05:56:41 2022] GET /identity/v3/role_assignments?user.id=8b2c3a376af245f2a0e347e2f08885a5&scope.project.id=6d7fd5ca249a4df5aba301beb23e2eb7&role.id=49e47cd157d743f6905f1db07ec878a7 => generated 259 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:56:41.643244 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ddd45f-afdb-4b87-a3fc-4f5939a1723f None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:56:41.685573 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:56:41.780729 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e1b3dd-c815-448b-8831-9da8f1936072 None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:56:41.801255 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:56:41.835501 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3b4f10-64b8-413c-b516-7a24d797983d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:56:41.835928 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3b4f10-64b8-413c-b516-7a24d797983d None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:41.836221 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3b4f10-64b8-413c-b516-7a24d797983d None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:41.837383 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3b4f10-64b8-413c-b516-7a24d797983d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:41.838075 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3b4f10-64b8-413c-b516-7a24d797983d None admin] RBAC: Adding query filter params (role.id=ba3c2c3cb98c424f9ece19903a59c53c, scope.project.id=6d7fd5ca249a4df5aba301beb23e2eb7, user.id=f16ca5f6955045a5811c25a8e6cc8055) {{(pid=81851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:56:41.839241 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3b4f10-64b8-413c-b516-7a24d797983d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:56:41.840749 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3b4f10-64b8-413c-b516-7a24d797983d None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:56:41.843188 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ddd45f-afdb-4b87-a3fc-4f5939a1723f None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:56:41.847607 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 48/94] 10.222.0.62 () {56 vars in 1374 bytes} [Wed Aug 24 05:56:41 2022] GET /identity/v3/role_assignments?user.id=f16ca5f6955045a5811c25a8e6cc8055&scope.project.id=6d7fd5ca249a4df5aba301beb23e2eb7&role.id=ba3c2c3cb98c424f9ece19903a59c53c => generated 259 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:56:41.895304 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3612aacf-5ea0-4fea-9216-b2cfabed3447 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:56:41.895949 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3612aacf-5ea0-4fea-9216-b2cfabed3447 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:41.896230 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3612aacf-5ea0-4fea-9216-b2cfabed3447 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:41.896549 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3612aacf-5ea0-4fea-9216-b2cfabed3447 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:41.978697 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3612aacf-5ea0-4fea-9216-b2cfabed3447 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:56:41.980109 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3612aacf-5ea0-4fea-9216-b2cfabed3447 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:56:42.009226 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:56:42.084568 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 47/95] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Aug 24 05:56:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 442 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 05:56:42.090418 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3b4f10-64b8-413c-b516-7a24d797983d None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:56:42.217666 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3b4f10-64b8-413c-b516-7a24d797983d None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:56:42.259605 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d2d50d-7276-40da-b5f0-a9c8df6a87e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:56:42.260178 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d2d50d-7276-40da-b5f0-a9c8df6a87e6 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:42.260456 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d2d50d-7276-40da-b5f0-a9c8df6a87e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:42.260756 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d2d50d-7276-40da-b5f0-a9c8df6a87e6 None admin] PATH_INFO: `/v3/projects/d8715c59754a4b35b2e19f4759824cf7` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:42.261892 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d2d50d-7276-40da-b5f0-a9c8df6a87e6 None admin] RBAC: Authorizing `identity:get_project(project_id=d8715c59754a4b35b2e19f4759824cf7)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:56:42.263345 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d2d50d-7276-40da-b5f0-a9c8df6a87e6 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:56:42.265297 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 49/96] 10.222.0.62 () {56 vars in 1186 bytes} [Wed Aug 24 05:56:42 2022] GET /identity/v3/projects/d8715c59754a4b35b2e19f4759824cf7 => generated 312 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:56:42.271188 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3612aacf-5ea0-4fea-9216-b2cfabed3447 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:56:42.390530 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3612aacf-5ea0-4fea-9216-b2cfabed3447 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:56:42.416449 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:56:42.433543 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9921c3-0850-4042-ad40-5016266f02d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:56:42.434160 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9921c3-0850-4042-ad40-5016266f02d7 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:42.434446 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9921c3-0850-4042-ad40-5016266f02d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:42.434724 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9921c3-0850-4042-ad40-5016266f02d7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:42.435453 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9921c3-0850-4042-ad40-5016266f02d7 None admin] RBAC: Adding query filter params (role.id=580aa7dbb38d4cff8f9d1bb8983f99ef, scope.project.id=d8715c59754a4b35b2e19f4759824cf7, user.id=f16ca5f6955045a5811c25a8e6cc8055) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:56:42.435765 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9921c3-0850-4042-ad40-5016266f02d7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:56:42.437506 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9921c3-0850-4042-ad40-5016266f02d7 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:56:42.445010 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 48/97] 10.222.0.62 () {56 vars in 1374 bytes} [Wed Aug 24 05:56:42 2022] GET /identity/v3/role_assignments?user.id=f16ca5f6955045a5811c25a8e6cc8055&scope.project.id=d8715c59754a4b35b2e19f4759824cf7&role.id=580aa7dbb38d4cff8f9d1bb8983f99ef => generated 259 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:56:43.586901 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9cc70c-2f4a-45e1-bf6c-bcec18884882 None None] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:43.587419 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9cc70c-2f4a-45e1-bf6c-bcec18884882 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:43.587419 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9cc70c-2f4a-45e1-bf6c-bcec18884882 None None] PATH_INFO: `/` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:43.589089 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 50/98] 10.222.0.62 () {54 vars in 927 bytes} [Wed Aug 24 05:56:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:56:43.595423 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72883c14-f393-4459-8472-bf0c79b868ea None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:43.595858 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72883c14-f393-4459-8472-bf0c79b868ea None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:43.595858 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72883c14-f393-4459-8472-bf0c79b868ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:43.628351 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-72883c14-f393-4459-8472-bf0c79b868ea None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:56:43.629600 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-72883c14-f393-4459-8472-bf0c79b868ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:56:43.769608 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 49/99] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Aug 24 05:56:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 05:56:44.322604 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9cc70c-2f4a-45e1-bf6c-bcec18884882 None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:56:44.366668 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c836567d-cb4c-440a-b41d-2b73a17ce0a9 None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:44.366972 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c836567d-cb4c-440a-b41d-2b73a17ce0a9 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:44.367226 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c836567d-cb4c-440a-b41d-2b73a17ce0a9 None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:44.368717 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 50/100] 10.222.0.62 () {54 vars in 927 bytes} [Wed Aug 24 05:56:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:56:44.373856 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d159a57-527f-4222-a864-ac4558b9be78 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:44.374139 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d159a57-527f-4222-a864-ac4558b9be78 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:44.374416 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d159a57-527f-4222-a864-ac4558b9be78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:44.392408 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:56:44.404754 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-8d159a57-527f-4222-a864-ac4558b9be78 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:56:44.405947 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-8d159a57-527f-4222-a864-ac4558b9be78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:56:44.490762 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 51/101] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Aug 24 05:56:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Aug 24 05:56:44.498135 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9cc70c-2f4a-45e1-bf6c-bcec18884882 None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:56:44.549595 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63987217-3047-46eb-ba58-85e740d42571 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:56:44.550748 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63987217-3047-46eb-ba58-85e740d42571 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:44.551027 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63987217-3047-46eb-ba58-85e740d42571 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:44.551296 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63987217-3047-46eb-ba58-85e740d42571 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:44.566032 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63987217-3047-46eb-ba58-85e740d42571 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:56:44.570728 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application [None req-63987217-3047-46eb-ba58-85e740d42571 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 05:56:44.570728 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:56:44.570728 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:56:44.570728 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:56:44.570728 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:56:44.570728 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:56:44.570728 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:56:44.570728 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:56:44.570728 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:56:44.570728 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:56:44.570728 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:56:44.570728 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:56:44.570728 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:56:44.570728 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:56:44.570728 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:56:44.570728 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:56:44.571623 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:56:44.571623 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:56:44.571623 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:56:44.571623 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:56:44.571623 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:56:44.571623 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:56:44.571623 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:56:44.571623 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:56:44.571623 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:56:44.571623 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:56:44.571623 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:56:44.571623 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application with Lock( Aug 24 05:56:44.571623 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:56:44.571623 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:56:44.571623 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:56:44.571623 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:56:44.572390 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:56:44.572390 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:56:44.572390 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:56:44.572390 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:56:44.572390 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:56:44.572390 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:56:44.572390 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:56:44.572390 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:56:44.572390 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:56:44.572390 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:56:44.572390 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 05:56:44.572390 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Aug 24 05:56:44.572390 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 51/102] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Aug 24 05:56:44 2022] GET /identity/v3/roles/member => generated 84 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:56:44.578098 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d159a57-527f-4222-a864-ac4558b9be78 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:56:44.578856 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e8b5dd-6b67-4664-b56a-2d613742b041 None None] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:44.579145 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e8b5dd-6b67-4664-b56a-2d613742b041 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:44.579424 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e8b5dd-6b67-4664-b56a-2d613742b041 None None] PATH_INFO: `/` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:44.580859 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 52/103] 10.222.0.62 () {54 vars in 927 bytes} [Wed Aug 24 05:56:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 05:56:44.585520 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b070af-66c5-4b91-95e8-40cf40d8a020 None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:44.585804 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b070af-66c5-4b91-95e8-40cf40d8a020 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:44.586079 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b070af-66c5-4b91-95e8-40cf40d8a020 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:44.621495 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-05b070af-66c5-4b91-95e8-40cf40d8a020 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:56:44.622702 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-05b070af-66c5-4b91-95e8-40cf40d8a020 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:56:44.663198 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:56:44.737851 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d159a57-527f-4222-a864-ac4558b9be78 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:56:44.778204 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7087e0f-0659-4ff9-98a0-c7468985ae34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:56:44.779167 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7087e0f-0659-4ff9-98a0-c7468985ae34 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:44.779592 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7087e0f-0659-4ff9-98a0-c7468985ae34 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:44.780006 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7087e0f-0659-4ff9-98a0-c7468985ae34 None admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:44.780845 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7087e0f-0659-4ff9-98a0-c7468985ae34 None admin] RBAC: Adding query filter params (name=member) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:56:44.781634 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7087e0f-0659-4ff9-98a0-c7468985ae34 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:56:44.783676 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7087e0f-0659-4ff9-98a0-c7468985ae34 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:56:44.791942 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 53/104] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Aug 24 05:56:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 05:56:44.794043 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 52/105] 10.222.0.62 () {56 vars in 1101 bytes} [Wed Aug 24 05:56:44 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:56:44.800033 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b070af-66c5-4b91-95e8-40cf40d8a020 None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:56:44.836349 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:56:44.903694 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b070af-66c5-4b91-95e8-40cf40d8a020 None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:56:44.939521 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b4bc70-dd27-4616-8362-c5e180d28e99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:56:44.940917 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b4bc70-dd27-4616-8362-c5e180d28e99 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:44.941205 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b4bc70-dd27-4616-8362-c5e180d28e99 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:44.941477 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b4bc70-dd27-4616-8362-c5e180d28e99 None admin] PATH_INFO: `/v3/users/f16ca5f6955045a5811c25a8e6cc8055` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:44.942617 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b4bc70-dd27-4616-8362-c5e180d28e99 None admin] RBAC: Authorizing `identity:get_user(user_id=f16ca5f6955045a5811c25a8e6cc8055)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:56:44.944199 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b4bc70-dd27-4616-8362-c5e180d28e99 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:56:44.945821 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 54/106] 10.222.0.62 () {56 vars in 1177 bytes} [Wed Aug 24 05:56:44 2022] GET /identity/v3/users/f16ca5f6955045a5811c25a8e6cc8055 => generated 275 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:56:44.951569 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7087e0f-0659-4ff9-98a0-c7468985ae34 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:56:45.037042 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b4bc70-dd27-4616-8362-c5e180d28e99 None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:56:45.067236 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7087e0f-0659-4ff9-98a0-c7468985ae34 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:56:45.082669 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:56:45.129549 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a65b7b9-3541-4019-89e5-08f4964e57e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:56:45.130148 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a65b7b9-3541-4019-89e5-08f4964e57e5 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:45.130430 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a65b7b9-3541-4019-89e5-08f4964e57e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:45.130704 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a65b7b9-3541-4019-89e5-08f4964e57e5 None admin] PATH_INFO: `/v3/projects/6d7fd5ca249a4df5aba301beb23e2eb7` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:45.131897 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a65b7b9-3541-4019-89e5-08f4964e57e5 None admin] RBAC: Authorizing `identity:get_project(project_id=6d7fd5ca249a4df5aba301beb23e2eb7)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:56:45.133481 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a65b7b9-3541-4019-89e5-08f4964e57e5 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:56:45.135306 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 53/107] 10.222.0.62 () {56 vars in 1186 bytes} [Wed Aug 24 05:56:44 2022] GET /identity/v3/projects/6d7fd5ca249a4df5aba301beb23e2eb7 => generated 298 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:56:45.136072 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a65b7b9-3541-4019-89e5-08f4964e57e5 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:56:45.162335 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b4bc70-dd27-4616-8362-c5e180d28e99 None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:56:45.164448 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:56:45.206978 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826934cd-cbad-4521-9768-a1af995257ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:56:45.207958 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826934cd-cbad-4521-9768-a1af995257ff None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:45.208431 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826934cd-cbad-4521-9768-a1af995257ff None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:45.208920 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826934cd-cbad-4521-9768-a1af995257ff None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:45.221260 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826934cd-cbad-4521-9768-a1af995257ff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:56:45.229051 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application [None req-826934cd-cbad-4521-9768-a1af995257ff None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 05:56:45.229051 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:56:45.229051 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:56:45.229051 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:56:45.229051 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:56:45.229051 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:56:45.229051 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:56:45.229051 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:56:45.229051 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:56:45.229051 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:56:45.229051 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:56:45.229051 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:56:45.229051 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:56:45.229051 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:56:45.229051 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:56:45.229051 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:56:45.229905 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:56:45.229905 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:56:45.229905 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:56:45.229905 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:56:45.229905 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:56:45.229905 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:56:45.229905 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:56:45.229905 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:56:45.229905 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:56:45.229905 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:56:45.229905 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:56:45.229905 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application with Lock( Aug 24 05:56:45.229905 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:56:45.229905 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:56:45.229905 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:56:45.229905 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:56:45.230652 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:56:45.230652 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:56:45.230652 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:56:45.230652 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:56:45.230652 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:56:45.230652 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:56:45.230652 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:56:45.230652 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:56:45.230652 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:56:45.230652 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:56:45.230652 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 05:56:45.230652 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Aug 24 05:56:45.233194 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 55/108] 10.222.0.62 () {56 vars in 1096 bytes} [Wed Aug 24 05:56:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 05:56:45.234010 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826934cd-cbad-4521-9768-a1af995257ff None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:56:45.307950 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a65b7b9-3541-4019-89e5-08f4964e57e5 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:56:45.349286 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826934cd-cbad-4521-9768-a1af995257ff None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:56:45.354826 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4fe2e8-1830-406d-8d17-acb942f477d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:56:45.355572 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4fe2e8-1830-406d-8d17-acb942f477d6 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:45.355950 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4fe2e8-1830-406d-8d17-acb942f477d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:45.356322 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4fe2e8-1830-406d-8d17-acb942f477d6 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:45.370551 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4fe2e8-1830-406d-8d17-acb942f477d6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:56:45.377399 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-aa4fe2e8-1830-406d-8d17-acb942f477d6 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 05:56:45.377399 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:56:45.377399 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:56:45.377399 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:56:45.377399 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:56:45.377399 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:56:45.377399 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:56:45.377399 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:56:45.377399 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:56:45.377399 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:56:45.377399 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:56:45.377399 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:56:45.377399 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:56:45.377399 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:56:45.377399 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:56:45.377399 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:56:45.378494 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:56:45.378494 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:56:45.378494 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:56:45.378494 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:56:45.378494 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:56:45.378494 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:56:45.378494 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:56:45.378494 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:56:45.378494 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:56:45.378494 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:56:45.378494 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:56:45.378494 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 24 05:56:45.378494 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:56:45.378494 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:56:45.378494 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:56:45.378494 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:56:45.379286 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:56:45.379286 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:56:45.379286 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:56:45.379286 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:56:45.379286 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:56:45.379286 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:56:45.379286 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:56:45.379286 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:56:45.379286 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:56:45.379286 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:56:45.379286 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 05:56:45.379286 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 24 05:56:45.379929 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 54/109] 10.222.0.62 () {56 vars in 1114 bytes} [Wed Aug 24 05:56:45 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:56:45.381610 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81b411a-bb3c-4005-baec-8c80c5dde4b8 None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:45.381962 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81b411a-bb3c-4005-baec-8c80c5dde4b8 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:45.382299 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81b411a-bb3c-4005-baec-8c80c5dde4b8 None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:45.384909 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 55/110] 10.222.0.62 () {54 vars in 927 bytes} [Wed Aug 24 05:56:45 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:56:45.385799 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81b411a-bb3c-4005-baec-8c80c5dde4b8 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:56:45.397857 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1568b31-a1c5-45e0-b2c3-b7c969c475c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:56:45.398720 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1568b31-a1c5-45e0-b2c3-b7c969c475c6 None admin] REQUEST_METHOD: `PUT` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:45.399139 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1568b31-a1c5-45e0-b2c3-b7c969c475c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:45.399568 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1568b31-a1c5-45e0-b2c3-b7c969c475c6 None admin] PATH_INFO: `/v3/projects/6d7fd5ca249a4df5aba301beb23e2eb7/users/f16ca5f6955045a5811c25a8e6cc8055/roles/580aa7dbb38d4cff8f9d1bb8983f99ef` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:45.402670 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1568b31-a1c5-45e0-b2c3-b7c969c475c6 None admin] RBAC: Authorizing `identity:create_grant(project_id=6d7fd5ca249a4df5aba301beb23e2eb7, user_id=f16ca5f6955045a5811c25a8e6cc8055, role_id=580aa7dbb38d4cff8f9d1bb8983f99ef)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:56:45.404779 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1568b31-a1c5-45e0-b2c3-b7c969c475c6 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:56:45.429981 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:56:45.492647 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 56/111] 10.222.0.62 () {58 vars in 1439 bytes} [Wed Aug 24 05:56:45 2022] PUT /identity/v3/projects/6d7fd5ca249a4df5aba301beb23e2eb7/users/f16ca5f6955045a5811c25a8e6cc8055/roles/580aa7dbb38d4cff8f9d1bb8983f99ef => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:56:45.493615 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1568b31-a1c5-45e0-b2c3-b7c969c475c6 None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:56:45.501708 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81b411a-bb3c-4005-baec-8c80c5dde4b8 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:56:45.538350 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:56:45.559682 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5aa1c9-951f-4e09-b16f-3155edc6d0f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:56:45.560311 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5aa1c9-951f-4e09-b16f-3155edc6d0f0 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:45.560836 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5aa1c9-951f-4e09-b16f-3155edc6d0f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:45.560940 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5aa1c9-951f-4e09-b16f-3155edc6d0f0 None admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:45.561552 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5aa1c9-951f-4e09-b16f-3155edc6d0f0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:56:45.561855 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5aa1c9-951f-4e09-b16f-3155edc6d0f0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:56:45.563273 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5aa1c9-951f-4e09-b16f-3155edc6d0f0 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:56:45.572803 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 56/112] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Aug 24 05:56:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:56:45.576576 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4dc347-0209-42d6-a9ee-1a5085fba255 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:45.576576 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4dc347-0209-42d6-a9ee-1a5085fba255 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:45.576576 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4dc347-0209-42d6-a9ee-1a5085fba255 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:45.595979 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-4b4dc347-0209-42d6-a9ee-1a5085fba255 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:56:45.598051 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-4b4dc347-0209-42d6-a9ee-1a5085fba255 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:56:45.628592 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1568b31-a1c5-45e0-b2c3-b7c969c475c6 None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:56:45.670537 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad4f50c-c191-49fd-b38e-49efbf0503a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:56:45.671162 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad4f50c-c191-49fd-b38e-49efbf0503a0 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:45.671446 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad4f50c-c191-49fd-b38e-49efbf0503a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:45.671722 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad4f50c-c191-49fd-b38e-49efbf0503a0 None admin] PATH_INFO: `/v3/roles` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:45.672387 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad4f50c-c191-49fd-b38e-49efbf0503a0 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:56:45.672871 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad4f50c-c191-49fd-b38e-49efbf0503a0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:56:45.674449 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad4f50c-c191-49fd-b38e-49efbf0503a0 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:56:45.679186 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 57/113] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Aug 24 05:56:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 05:56:45.680648 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b785c4-d78b-41bf-ade7-62278a89fa4f None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:45.681078 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b785c4-d78b-41bf-ade7-62278a89fa4f None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:45.681524 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b785c4-d78b-41bf-ade7-62278a89fa4f None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:45.682984 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 58/114] 10.222.0.62 () {54 vars in 927 bytes} [Wed Aug 24 05:56:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:56:45.684177 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0eedcf0-7eb4-4fe3-a3ad-92dd4e960070 None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:45.684664 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0eedcf0-7eb4-4fe3-a3ad-92dd4e960070 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:45.685120 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0eedcf0-7eb4-4fe3-a3ad-92dd4e960070 None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:45.690991 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 59/115] 10.222.0.62 () {54 vars in 927 bytes} [Wed Aug 24 05:56:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:56:45.691144 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 57/116] 10.222.0.62 () {56 vars in 1111 bytes} [Wed Aug 24 05:56:45 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:56:45.691144 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad4f50c-c191-49fd-b38e-49efbf0503a0 None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:56:45.692626 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b2b2ea-01cf-4246-a731-a7f01b504011 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:45.693094 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b2b2ea-01cf-4246-a731-a7f01b504011 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:45.693546 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b2b2ea-01cf-4246-a731-a7f01b504011 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:45.712417 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-b3b2b2ea-01cf-4246-a731-a7f01b504011 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:56:45.713834 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-b3b2b2ea-01cf-4246-a731-a7f01b504011 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:56:45.737651 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:56:45.785707 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:56:45.785889 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 60/117] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Aug 24 05:56:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 05:56:45.786998 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f735057-4c12-4a73-a628-d5134cd8c5cb None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:45.787530 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f735057-4c12-4a73-a628-d5134cd8c5cb None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:45.787986 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f735057-4c12-4a73-a628-d5134cd8c5cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:45.806949 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-2f735057-4c12-4a73-a628-d5134cd8c5cb None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:56:45.808681 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-2f735057-4c12-4a73-a628-d5134cd8c5cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:56:45.820301 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad4f50c-c191-49fd-b38e-49efbf0503a0 None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:56:45.887368 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 61/118] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Aug 24 05:56:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 05:56:45.888534 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f735057-4c12-4a73-a628-d5134cd8c5cb None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:56:45.897319 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb597e28-33a5-4556-8633-43b1e607194e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:56:45.897985 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb597e28-33a5-4556-8633-43b1e607194e None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:45.898273 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb597e28-33a5-4556-8633-43b1e607194e None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:45.898563 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb597e28-33a5-4556-8633-43b1e607194e None admin] PATH_INFO: `/v3/users/8b2c3a376af245f2a0e347e2f08885a5` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:45.899865 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb597e28-33a5-4556-8633-43b1e607194e None admin] RBAC: Authorizing `identity:get_user(user_id=8b2c3a376af245f2a0e347e2f08885a5)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:56:45.901481 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb597e28-33a5-4556-8633-43b1e607194e None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:56:45.903087 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 58/119] 10.222.0.62 () {56 vars in 1177 bytes} [Wed Aug 24 05:56:45 2022] GET /identity/v3/users/8b2c3a376af245f2a0e347e2f08885a5 => generated 247 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:56:45.912564 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb597e28-33a5-4556-8633-43b1e607194e None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:56:46.045862 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:56:46.071830 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f735057-4c12-4a73-a628-d5134cd8c5cb None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:56:46.098118 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb597e28-33a5-4556-8633-43b1e607194e None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:56:46.133101 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23e0d48-5078-44e8-8ed7-a909910b360c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:56:46.133738 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23e0d48-5078-44e8-8ed7-a909910b360c None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:46.134010 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23e0d48-5078-44e8-8ed7-a909910b360c None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:46.134279 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23e0d48-5078-44e8-8ed7-a909910b360c None admin] PATH_INFO: `/v3/users/f16ca5f6955045a5811c25a8e6cc8055` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:46.135490 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23e0d48-5078-44e8-8ed7-a909910b360c None admin] RBAC: Authorizing `identity:get_user(user_id=f16ca5f6955045a5811c25a8e6cc8055)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:56:46.136969 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23e0d48-5078-44e8-8ed7-a909910b360c None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:56:46.144224 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 62/120] 10.222.0.62 () {56 vars in 1177 bytes} [Wed Aug 24 05:56:45 2022] GET /identity/v3/users/f16ca5f6955045a5811c25a8e6cc8055 => generated 275 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:56:46.146527 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23e0d48-5078-44e8-8ed7-a909910b360c None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:56:46.159273 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d5c9b5-f408-45e3-bd88-065290c16c6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:56:46.160150 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d5c9b5-f408-45e3-bd88-065290c16c6c None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:46.161807 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d5c9b5-f408-45e3-bd88-065290c16c6c None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:46.162268 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d5c9b5-f408-45e3-bd88-065290c16c6c None admin] PATH_INFO: `/v3/projects/6d7fd5ca249a4df5aba301beb23e2eb7` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:46.164256 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d5c9b5-f408-45e3-bd88-065290c16c6c None admin] RBAC: Authorizing `identity:get_project(project_id=6d7fd5ca249a4df5aba301beb23e2eb7)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:56:46.166420 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d5c9b5-f408-45e3-bd88-065290c16c6c None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:56:46.169295 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 59/121] 10.222.0.62 () {56 vars in 1186 bytes} [Wed Aug 24 05:56:45 2022] GET /identity/v3/projects/6d7fd5ca249a4df5aba301beb23e2eb7 => generated 298 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:56:46.170708 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d5c9b5-f408-45e3-bd88-065290c16c6c None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:56:46.201205 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:56:46.264279 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23e0d48-5078-44e8-8ed7-a909910b360c None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:56:46.300579 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea17cb5-5d24-4d04-8505-aeb73594ee6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:56:46.301140 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea17cb5-5d24-4d04-8505-aeb73594ee6a None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:46.301421 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea17cb5-5d24-4d04-8505-aeb73594ee6a None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:46.301696 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea17cb5-5d24-4d04-8505-aeb73594ee6a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:46.306107 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d5c9b5-f408-45e3-bd88-065290c16c6c None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:56:46.312438 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea17cb5-5d24-4d04-8505-aeb73594ee6a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:56:46.317906 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-5ea17cb5-5d24-4d04-8505-aeb73594ee6a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 05:56:46.317906 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:56:46.317906 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:56:46.317906 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:56:46.317906 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:56:46.317906 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:56:46.317906 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:56:46.317906 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:56:46.317906 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:56:46.317906 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:56:46.317906 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:56:46.317906 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:56:46.317906 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:56:46.317906 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:56:46.317906 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:56:46.317906 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:56:46.318770 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:56:46.318770 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:56:46.318770 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:56:46.318770 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:56:46.318770 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:56:46.318770 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:56:46.318770 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:56:46.318770 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:56:46.318770 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:56:46.318770 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:56:46.318770 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:56:46.318770 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 24 05:56:46.318770 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:56:46.318770 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:56:46.318770 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:56:46.318770 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:56:46.319573 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:56:46.319573 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:56:46.319573 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:56:46.319573 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:56:46.319573 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:56:46.319573 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:56:46.319573 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:56:46.319573 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:56:46.319573 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:56:46.319573 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:56:46.319573 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 05:56:46.319573 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 24 05:56:46.319573 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 63/122] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Aug 24 05:56:46 2022] GET /identity/v3/roles/reader => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:56:46.320218 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea17cb5-5d24-4d04-8505-aeb73594ee6a None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:56:46.338789 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:56:46.344738 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a0cffe-4bdb-461c-b044-60fe5fe930d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:56:46.345376 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a0cffe-4bdb-461c-b044-60fe5fe930d5 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:46.345656 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a0cffe-4bdb-461c-b044-60fe5fe930d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:46.354645 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a0cffe-4bdb-461c-b044-60fe5fe930d5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:46.354645 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a0cffe-4bdb-461c-b044-60fe5fe930d5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:56:46.354645 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a0cffe-4bdb-461c-b044-60fe5fe930d5 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:56:46.354645 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 60/123] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Aug 24 05:56:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:56:46.354645 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a0cffe-4bdb-461c-b044-60fe5fe930d5 None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:56:46.423438 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea17cb5-5d24-4d04-8505-aeb73594ee6a None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:56:46.456303 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:56:46.458680 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4432279e-0ab3-40da-ae46-1cff8b8079f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:56:46.459315 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4432279e-0ab3-40da-ae46-1cff8b8079f4 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:46.459601 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4432279e-0ab3-40da-ae46-1cff8b8079f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:46.460114 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4432279e-0ab3-40da-ae46-1cff8b8079f4 None admin] PATH_INFO: `/v3/projects/6d7fd5ca249a4df5aba301beb23e2eb7` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:46.461021 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4432279e-0ab3-40da-ae46-1cff8b8079f4 None admin] RBAC: Authorizing `identity:get_project(project_id=6d7fd5ca249a4df5aba301beb23e2eb7)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:56:46.462486 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4432279e-0ab3-40da-ae46-1cff8b8079f4 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:56:46.464279 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 64/124] 10.222.0.62 () {56 vars in 1186 bytes} [Wed Aug 24 05:56:46 2022] GET /identity/v3/projects/6d7fd5ca249a4df5aba301beb23e2eb7 => generated 298 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:56:46.464901 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4432279e-0ab3-40da-ae46-1cff8b8079f4 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:56:46.473497 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a0cffe-4bdb-461c-b044-60fe5fe930d5 None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:56:46.510788 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693ba8db-4682-451b-a03c-1d79e9e03427 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:56:46.511459 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693ba8db-4682-451b-a03c-1d79e9e03427 None admin] REQUEST_METHOD: `PUT` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:46.511727 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693ba8db-4682-451b-a03c-1d79e9e03427 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:46.512001 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693ba8db-4682-451b-a03c-1d79e9e03427 None admin] PATH_INFO: `/v3/projects/6d7fd5ca249a4df5aba301beb23e2eb7/users/8b2c3a376af245f2a0e347e2f08885a5/roles/49e47cd157d743f6905f1db07ec878a7` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:46.513914 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693ba8db-4682-451b-a03c-1d79e9e03427 None admin] RBAC: Authorizing `identity:create_grant(project_id=6d7fd5ca249a4df5aba301beb23e2eb7, user_id=8b2c3a376af245f2a0e347e2f08885a5, role_id=49e47cd157d743f6905f1db07ec878a7)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:56:46.515354 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693ba8db-4682-451b-a03c-1d79e9e03427 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:56:46.551571 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:56:46.567150 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4432279e-0ab3-40da-ae46-1cff8b8079f4 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:56:46.599879 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e50345-e707-41e4-83ee-cdd33995edc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:56:46.600612 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e50345-e707-41e4-83ee-cdd33995edc8 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:46.600834 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e50345-e707-41e4-83ee-cdd33995edc8 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:46.601100 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e50345-e707-41e4-83ee-cdd33995edc8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:46.607273 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 61/125] 10.222.0.62 () {58 vars in 1439 bytes} [Wed Aug 24 05:56:46 2022] PUT /identity/v3/projects/6d7fd5ca249a4df5aba301beb23e2eb7/users/8b2c3a376af245f2a0e347e2f08885a5/roles/49e47cd157d743f6905f1db07ec878a7 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:56:46.613001 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693ba8db-4682-451b-a03c-1d79e9e03427 None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:56:46.623678 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e50345-e707-41e4-83ee-cdd33995edc8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:56:46.633903 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-64e50345-e707-41e4-83ee-cdd33995edc8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 05:56:46.633903 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:56:46.633903 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:56:46.633903 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:56:46.633903 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:56:46.633903 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:56:46.633903 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:56:46.633903 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:56:46.633903 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:56:46.633903 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:56:46.633903 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:56:46.633903 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:56:46.633903 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:56:46.633903 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:56:46.633903 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:56:46.633903 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:56:46.637621 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:56:46.637621 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:56:46.637621 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:56:46.637621 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:56:46.637621 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:56:46.637621 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:56:46.637621 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:56:46.637621 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:56:46.637621 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:56:46.637621 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:56:46.637621 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:56:46.637621 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 24 05:56:46.637621 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:56:46.637621 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:56:46.637621 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:56:46.637621 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:56:46.638392 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:56:46.638392 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:56:46.638392 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:56:46.638392 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:56:46.638392 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:56:46.638392 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:56:46.638392 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:56:46.638392 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:56:46.638392 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:56:46.638392 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:56:46.638392 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 05:56:46.638392 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 24 05:56:46.639878 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 65/126] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Aug 24 05:56:46 2022] GET /identity/v3/roles/member => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:56:46.641357 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e50345-e707-41e4-83ee-cdd33995edc8 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:56:46.758293 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693ba8db-4682-451b-a03c-1d79e9e03427 None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:56:46.792190 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e50345-e707-41e4-83ee-cdd33995edc8 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:56:46.804454 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4546be2-07e5-41a2-b5aa-d31b8f2e4d39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:56:46.805606 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4546be2-07e5-41a2-b5aa-d31b8f2e4d39 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:46.806027 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4546be2-07e5-41a2-b5aa-d31b8f2e4d39 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:46.806635 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4546be2-07e5-41a2-b5aa-d31b8f2e4d39 None admin] PATH_INFO: `/v3/roles` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:46.807804 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4546be2-07e5-41a2-b5aa-d31b8f2e4d39 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:56:46.808413 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4546be2-07e5-41a2-b5aa-d31b8f2e4d39 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:56:46.810239 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4546be2-07e5-41a2-b5aa-d31b8f2e4d39 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:56:46.821958 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 62/127] 10.222.0.62 () {56 vars in 1101 bytes} [Wed Aug 24 05:56:46 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:56:46.823403 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4546be2-07e5-41a2-b5aa-d31b8f2e4d39 None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:56:46.827944 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf0691a-b811-4221-96dd-88852395b39c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:56:46.829075 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf0691a-b811-4221-96dd-88852395b39c None admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:46.829075 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf0691a-b811-4221-96dd-88852395b39c None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:46.829236 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf0691a-b811-4221-96dd-88852395b39c None admin] PATH_INFO: `/v3/projects` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:46.829818 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf0691a-b811-4221-96dd-88852395b39c None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:56:46.831823 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf0691a-b811-4221-96dd-88852395b39c None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:56:46.848833 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:56:46.851318 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:56:46.911509 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 66/128] 10.222.0.62 () {60 vars in 1141 bytes} [Wed Aug 24 05:56:46 2022] POST /identity/v3/projects => generated 302 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:56:46.916655 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf0691a-b811-4221-96dd-88852395b39c None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:56:47.003580 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4546be2-07e5-41a2-b5aa-d31b8f2e4d39 None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:56:47.062742 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db3d080-a44c-4955-b13d-d0ba4fdef14e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:56:47.063844 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db3d080-a44c-4955-b13d-d0ba4fdef14e None admin] REQUEST_METHOD: `PUT` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:47.064337 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db3d080-a44c-4955-b13d-d0ba4fdef14e None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:47.064834 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db3d080-a44c-4955-b13d-d0ba4fdef14e None admin] PATH_INFO: `/v3/projects/6d7fd5ca249a4df5aba301beb23e2eb7/users/f16ca5f6955045a5811c25a8e6cc8055/roles/ba3c2c3cb98c424f9ece19903a59c53c` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:47.068194 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db3d080-a44c-4955-b13d-d0ba4fdef14e None admin] RBAC: Authorizing `identity:create_grant(project_id=6d7fd5ca249a4df5aba301beb23e2eb7, user_id=f16ca5f6955045a5811c25a8e6cc8055, role_id=ba3c2c3cb98c424f9ece19903a59c53c)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:56:47.070102 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db3d080-a44c-4955-b13d-d0ba4fdef14e None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:56:47.119987 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf0691a-b811-4221-96dd-88852395b39c None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:56:47.161754 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f3541c-aec5-4535-9c83-4f5555a9dc41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:56:47.162377 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f3541c-aec5-4535-9c83-4f5555a9dc41 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:47.162659 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f3541c-aec5-4535-9c83-4f5555a9dc41 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:47.162935 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f3541c-aec5-4535-9c83-4f5555a9dc41 None admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:47.163693 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f3541c-aec5-4535-9c83-4f5555a9dc41 None admin] RBAC: Adding query filter params (name=member) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:56:47.164002 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f3541c-aec5-4535-9c83-4f5555a9dc41 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:56:47.165415 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f3541c-aec5-4535-9c83-4f5555a9dc41 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:56:47.176104 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 67/129] 10.222.0.62 () {56 vars in 1101 bytes} [Wed Aug 24 05:56:46 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:56:47.177582 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f3541c-aec5-4535-9c83-4f5555a9dc41 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:56:47.182324 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 63/130] 10.222.0.62 () {58 vars in 1439 bytes} [Wed Aug 24 05:56:46 2022] PUT /identity/v3/projects/6d7fd5ca249a4df5aba301beb23e2eb7/users/f16ca5f6955045a5811c25a8e6cc8055/roles/ba3c2c3cb98c424f9ece19903a59c53c => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:56:47.183487 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db3d080-a44c-4955-b13d-d0ba4fdef14e None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:56:47.294483 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:56:47.342688 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:56:47.366309 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f3541c-aec5-4535-9c83-4f5555a9dc41 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:56:47.419812 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97994e6-64f8-49ba-a8eb-13512162f8b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:56:47.420742 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97994e6-64f8-49ba-a8eb-13512162f8b4 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:47.421033 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97994e6-64f8-49ba-a8eb-13512162f8b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:47.421315 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97994e6-64f8-49ba-a8eb-13512162f8b4 None admin] PATH_INFO: `/v3/users/d88bdbc9f42241d9bac9843db6e7ef02` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:47.434440 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db3d080-a44c-4955-b13d-d0ba4fdef14e None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:56:47.447957 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97994e6-64f8-49ba-a8eb-13512162f8b4 None admin] RBAC: Authorizing `identity:get_user(user_id=d88bdbc9f42241d9bac9843db6e7ef02)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:56:47.450072 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97994e6-64f8-49ba-a8eb-13512162f8b4 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:56:47.452393 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 68/131] 10.222.0.62 () {56 vars in 1177 bytes} [Wed Aug 24 05:56:47 2022] GET /identity/v3/users/d88bdbc9f42241d9bac9843db6e7ef02 => generated 289 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:56:47.459085 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97994e6-64f8-49ba-a8eb-13512162f8b4 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:56:47.513421 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f46f3f-bb67-49bb-92fe-f88d21fe93d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:56:47.514070 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f46f3f-bb67-49bb-92fe-f88d21fe93d7 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:47.514351 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f46f3f-bb67-49bb-92fe-f88d21fe93d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:47.514626 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f46f3f-bb67-49bb-92fe-f88d21fe93d7 None admin] PATH_INFO: `/v3/users/f16ca5f6955045a5811c25a8e6cc8055` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:47.516034 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f46f3f-bb67-49bb-92fe-f88d21fe93d7 None admin] RBAC: Authorizing `identity:get_user(user_id=f16ca5f6955045a5811c25a8e6cc8055)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:56:47.519197 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f46f3f-bb67-49bb-92fe-f88d21fe93d7 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:56:47.522507 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 64/132] 10.222.0.62 () {56 vars in 1177 bytes} [Wed Aug 24 05:56:47 2022] GET /identity/v3/users/f16ca5f6955045a5811c25a8e6cc8055 => generated 275 bytes in 339 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:56:47.528343 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f46f3f-bb67-49bb-92fe-f88d21fe93d7 None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:56:47.610186 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97994e6-64f8-49ba-a8eb-13512162f8b4 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:56:47.635502 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:56:47.657155 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55da36eb-f151-4374-ad34-8c3747bfea11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:56:47.658122 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55da36eb-f151-4374-ad34-8c3747bfea11 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:47.659882 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55da36eb-f151-4374-ad34-8c3747bfea11 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:47.660325 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55da36eb-f151-4374-ad34-8c3747bfea11 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:47.706196 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f46f3f-bb67-49bb-92fe-f88d21fe93d7 None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:56:47.748691 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b495394-551e-4d1d-b051-d37b4694b858 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:56:47.750354 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b495394-551e-4d1d-b051-d37b4694b858 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:47.750637 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b495394-551e-4d1d-b051-d37b4694b858 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:47.750908 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b495394-551e-4d1d-b051-d37b4694b858 None admin] PATH_INFO: `/v3/projects/d8715c59754a4b35b2e19f4759824cf7` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:47.754448 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b495394-551e-4d1d-b051-d37b4694b858 None admin] RBAC: Authorizing `identity:get_project(project_id=d8715c59754a4b35b2e19f4759824cf7)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:56:47.757101 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b495394-551e-4d1d-b051-d37b4694b858 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:56:47.758787 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 65/133] 10.222.0.62 () {56 vars in 1186 bytes} [Wed Aug 24 05:56:47 2022] GET /identity/v3/projects/d8715c59754a4b35b2e19f4759824cf7 => generated 312 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:56:47.768158 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c5748d-1db4-4233-8308-c75e146e5bd4 None None] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:47.768534 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c5748d-1db4-4233-8308-c75e146e5bd4 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:47.768803 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c5748d-1db4-4233-8308-c75e146e5bd4 None None] PATH_INFO: `/` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:47.770320 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 66/134] 10.222.0.62 () {54 vars in 927 bytes} [Wed Aug 24 05:56:47 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:56:47.771181 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c5748d-1db4-4233-8308-c75e146e5bd4 None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:56:47.790349 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:56:47.814711 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55da36eb-f151-4374-ad34-8c3747bfea11 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:56:47.816877 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55da36eb-f151-4374-ad34-8c3747bfea11 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:56:47.928115 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 69/135] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Aug 24 05:56:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 470 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 24 05:56:47.933873 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fced9c-37d9-4a78-b0eb-7db550830428 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:47.934160 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fced9c-37d9-4a78-b0eb-7db550830428 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:47.934437 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fced9c-37d9-4a78-b0eb-7db550830428 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:47.954099 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-70fced9c-37d9-4a78-b0eb-7db550830428 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:56:47.955319 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-70fced9c-37d9-4a78-b0eb-7db550830428 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:56:47.968885 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:56:47.998113 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c5748d-1db4-4233-8308-c75e146e5bd4 None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:56:48.026645 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 70/136] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Aug 24 05:56:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 05:56:48.027628 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fced9c-37d9-4a78-b0eb-7db550830428 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:56:48.059295 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c33fc1-aff7-4ade-b062-70a9dd906c36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:56:48.059977 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c33fc1-aff7-4ade-b062-70a9dd906c36 None admin] REQUEST_METHOD: `PUT` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:48.060264 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c33fc1-aff7-4ade-b062-70a9dd906c36 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:48.060748 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c33fc1-aff7-4ade-b062-70a9dd906c36 None admin] PATH_INFO: `/v3/projects/d8715c59754a4b35b2e19f4759824cf7/users/f16ca5f6955045a5811c25a8e6cc8055/roles/580aa7dbb38d4cff8f9d1bb8983f99ef` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:48.062832 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c33fc1-aff7-4ade-b062-70a9dd906c36 None admin] RBAC: Authorizing `identity:create_grant(project_id=d8715c59754a4b35b2e19f4759824cf7, user_id=f16ca5f6955045a5811c25a8e6cc8055, role_id=580aa7dbb38d4cff8f9d1bb8983f99ef)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:56:48.064479 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c33fc1-aff7-4ade-b062-70a9dd906c36 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:56:48.132676 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fced9c-37d9-4a78-b0eb-7db550830428 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:56:48.167772 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797d94e2-8cac-46d7-8375-5ab805894500 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:56:48.168406 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797d94e2-8cac-46d7-8375-5ab805894500 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:48.168884 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797d94e2-8cac-46d7-8375-5ab805894500 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:48.169167 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797d94e2-8cac-46d7-8375-5ab805894500 None admin] PATH_INFO: `/v3/projects/6d7fd5ca249a4df5aba301beb23e2eb7` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:48.170551 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797d94e2-8cac-46d7-8375-5ab805894500 None admin] RBAC: Authorizing `identity:get_project(project_id=6d7fd5ca249a4df5aba301beb23e2eb7)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:56:48.172456 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797d94e2-8cac-46d7-8375-5ab805894500 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:56:48.174591 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 71/137] 10.222.0.62 () {56 vars in 1186 bytes} [Wed Aug 24 05:56:48 2022] GET /identity/v3/projects/6d7fd5ca249a4df5aba301beb23e2eb7 => generated 298 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:56:48.175956 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 67/138] 10.222.0.62 () {58 vars in 1439 bytes} [Wed Aug 24 05:56:47 2022] PUT /identity/v3/projects/d8715c59754a4b35b2e19f4759824cf7/users/f16ca5f6955045a5811c25a8e6cc8055/roles/580aa7dbb38d4cff8f9d1bb8983f99ef => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:56:48.186445 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797d94e2-8cac-46d7-8375-5ab805894500 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:56:48.198948 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:56:48.290757 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797d94e2-8cac-46d7-8375-5ab805894500 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:56:48.329036 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c30778a-f6e7-460e-9784-73785f4a15e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:56:48.329652 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c30778a-f6e7-460e-9784-73785f4a15e2 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:48.329930 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c30778a-f6e7-460e-9784-73785f4a15e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:48.330234 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c30778a-f6e7-460e-9784-73785f4a15e2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:48.331461 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c30778a-f6e7-460e-9784-73785f4a15e2 None admin] RBAC: Adding query filter params (role.id=e2f40474c6de4f5fadfa7efbe4087971, scope.project.id=6d7fd5ca249a4df5aba301beb23e2eb7, user.id=d88bdbc9f42241d9bac9843db6e7ef02) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:56:48.331769 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c30778a-f6e7-460e-9784-73785f4a15e2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:56:48.333302 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c30778a-f6e7-460e-9784-73785f4a15e2 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:56:48.339336 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 72/139] 10.222.0.62 () {56 vars in 1374 bytes} [Wed Aug 24 05:56:48 2022] GET /identity/v3/role_assignments?user.id=d88bdbc9f42241d9bac9843db6e7ef02&scope.project.id=6d7fd5ca249a4df5aba301beb23e2eb7&role.id=e2f40474c6de4f5fadfa7efbe4087971 => generated 259 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:56:48.500215 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c33fc1-aff7-4ade-b062-70a9dd906c36 None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:56:48.583269 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:56:48.632273 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c33fc1-aff7-4ade-b062-70a9dd906c36 None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:56:48.679144 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bb555e-a206-4ac3-9c2e-d71a96e740cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:56:48.680020 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bb555e-a206-4ac3-9c2e-d71a96e740cf None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:48.680476 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bb555e-a206-4ac3-9c2e-d71a96e740cf None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:48.680908 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bb555e-a206-4ac3-9c2e-d71a96e740cf None admin] PATH_INFO: `/v3/roles/member` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:48.699460 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79bb555e-a206-4ac3-9c2e-d71a96e740cf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:56:48.704994 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application [None req-79bb555e-a206-4ac3-9c2e-d71a96e740cf None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 05:56:48.704994 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:56:48.704994 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:56:48.704994 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:56:48.704994 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:56:48.704994 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:56:48.704994 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:56:48.704994 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:56:48.704994 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:56:48.704994 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:56:48.704994 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:56:48.704994 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:56:48.704994 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:56:48.704994 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:56:48.704994 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:56:48.704994 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:56:48.706071 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:56:48.706071 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:56:48.706071 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:56:48.706071 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:56:48.706071 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:56:48.706071 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:56:48.706071 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:56:48.706071 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:56:48.706071 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:56:48.706071 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:56:48.706071 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:56:48.706071 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application with Lock( Aug 24 05:56:48.706071 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:56:48.706071 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:56:48.706071 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:56:48.706071 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:56:48.707009 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:56:48.707009 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:56:48.707009 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:56:48.707009 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:56:48.707009 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:56:48.707009 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:56:48.707009 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:56:48.707009 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:56:48.707009 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:56:48.707009 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:56:48.707009 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 05:56:48.707009 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Aug 24 05:56:48.709014 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 68/140] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Aug 24 05:56:48 2022] GET /identity/v3/roles/member => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:56:48.714867 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c30778a-f6e7-460e-9784-73785f4a15e2 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:56:48.813042 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:56:48.843440 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c30778a-f6e7-460e-9784-73785f4a15e2 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:56:48.888433 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186361c8-9f52-4e6e-928c-da8135d7c171 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:56:48.889078 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186361c8-9f52-4e6e-928c-da8135d7c171 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:48.889359 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186361c8-9f52-4e6e-928c-da8135d7c171 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:48.889636 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186361c8-9f52-4e6e-928c-da8135d7c171 None admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:48.890301 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186361c8-9f52-4e6e-928c-da8135d7c171 None admin] RBAC: Adding query filter params (name=member) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:56:48.890604 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186361c8-9f52-4e6e-928c-da8135d7c171 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:56:48.892014 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186361c8-9f52-4e6e-928c-da8135d7c171 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:56:48.904584 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 73/141] 10.222.0.62 () {56 vars in 1101 bytes} [Wed Aug 24 05:56:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:56:48.914830 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bb555e-a206-4ac3-9c2e-d71a96e740cf None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:56:49.054976 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bb555e-a206-4ac3-9c2e-d71a96e740cf None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:56:49.096774 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:56:49.106002 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759bdcec-2d03-43f8-b9d7-5cf91d0308d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:56:49.116593 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759bdcec-2d03-43f8-b9d7-5cf91d0308d5 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:49.116593 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759bdcec-2d03-43f8-b9d7-5cf91d0308d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:49.116593 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759bdcec-2d03-43f8-b9d7-5cf91d0308d5 None admin] PATH_INFO: `/v3/users/f16ca5f6955045a5811c25a8e6cc8055` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:49.116593 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759bdcec-2d03-43f8-b9d7-5cf91d0308d5 None admin] RBAC: Authorizing `identity:get_user(user_id=f16ca5f6955045a5811c25a8e6cc8055)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:56:49.116593 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759bdcec-2d03-43f8-b9d7-5cf91d0308d5 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:56:49.116593 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 69/142] 10.222.0.62 () {56 vars in 1177 bytes} [Wed Aug 24 05:56:48 2022] GET /identity/v3/users/f16ca5f6955045a5811c25a8e6cc8055 => generated 275 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:56:49.118531 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186361c8-9f52-4e6e-928c-da8135d7c171 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:56:49.232114 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186361c8-9f52-4e6e-928c-da8135d7c171 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:56:49.275898 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a776535c-af53-46e8-8703-e52ac5610b53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:56:49.276543 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a776535c-af53-46e8-8703-e52ac5610b53 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:49.276827 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a776535c-af53-46e8-8703-e52ac5610b53 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:49.277097 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a776535c-af53-46e8-8703-e52ac5610b53 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:49.288172 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:56:49.355109 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a776535c-af53-46e8-8703-e52ac5610b53 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:56:49.356488 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a776535c-af53-46e8-8703-e52ac5610b53 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:56:49.414468 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e980cf-5199-4e0d-a862-a189b5ea91dc None None] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:49.415183 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e980cf-5199-4e0d-a862-a189b5ea91dc None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:49.415664 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e980cf-5199-4e0d-a862-a189b5ea91dc None None] PATH_INFO: `/` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:49.420768 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 70/143] 10.222.0.62 () {54 vars in 927 bytes} [Wed Aug 24 05:56:49 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:56:49.426833 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3294bed5-47fb-47e6-8ea0-0fed27649fc9 None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:49.427414 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3294bed5-47fb-47e6-8ea0-0fed27649fc9 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:49.427908 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3294bed5-47fb-47e6-8ea0-0fed27649fc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:49.468745 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-3294bed5-47fb-47e6-8ea0-0fed27649fc9 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:56:49.470322 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-3294bed5-47fb-47e6-8ea0-0fed27649fc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:56:49.507878 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 74/144] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Aug 24 05:56:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 05:56:49.514062 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a776535c-af53-46e8-8703-e52ac5610b53 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:56:49.566912 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:56:49.639221 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 71/145] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Aug 24 05:56:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 05:56:49.654931 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a776535c-af53-46e8-8703-e52ac5610b53 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:56:49.691411 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b295cce-af24-4e56-99e3-d67336b0bbff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:56:49.692046 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b295cce-af24-4e56-99e3-d67336b0bbff None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:49.692325 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b295cce-af24-4e56-99e3-d67336b0bbff None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:49.692634 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b295cce-af24-4e56-99e3-d67336b0bbff None admin] PATH_INFO: `/v3/projects/6d7fd5ca249a4df5aba301beb23e2eb7` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:49.693755 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b295cce-af24-4e56-99e3-d67336b0bbff None admin] RBAC: Authorizing `identity:get_project(project_id=6d7fd5ca249a4df5aba301beb23e2eb7)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:56:49.695191 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b295cce-af24-4e56-99e3-d67336b0bbff None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:56:49.696871 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 75/146] 10.222.0.62 () {56 vars in 1186 bytes} [Wed Aug 24 05:56:49 2022] GET /identity/v3/projects/6d7fd5ca249a4df5aba301beb23e2eb7 => generated 298 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:56:49.702768 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3294bed5-47fb-47e6-8ea0-0fed27649fc9 None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:56:49.791396 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8fd69d-bae3-45c9-a9ab-e18beb045605 None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:49.791733 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8fd69d-bae3-45c9-a9ab-e18beb045605 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:49.791943 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8fd69d-bae3-45c9-a9ab-e18beb045605 None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:49.793621 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 76/147] 10.222.0.62 () {54 vars in 927 bytes} [Wed Aug 24 05:56:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 05:56:49.801250 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4b09b5-0913-4b00-88f0-2044bbfefd16 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:49.801554 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4b09b5-0913-4b00-88f0-2044bbfefd16 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:49.801802 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4b09b5-0913-4b00-88f0-2044bbfefd16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:49.824013 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-ee4b09b5-0913-4b00-88f0-2044bbfefd16 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:56:49.825790 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-ee4b09b5-0913-4b00-88f0-2044bbfefd16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:56:49.830562 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3294bed5-47fb-47e6-8ea0-0fed27649fc9 None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:56:49.889926 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:56:49.902630 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 77/148] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Aug 24 05:56:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 05:56:49.914081 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73aaf4cb-0c9b-469a-8602-050b51be9854 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:56:49.914694 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73aaf4cb-0c9b-469a-8602-050b51be9854 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:49.914968 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73aaf4cb-0c9b-469a-8602-050b51be9854 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:49.915477 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73aaf4cb-0c9b-469a-8602-050b51be9854 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:49.915950 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73aaf4cb-0c9b-469a-8602-050b51be9854 None admin] RBAC: Adding query filter params (role.id=580aa7dbb38d4cff8f9d1bb8983f99ef, scope.project.id=6d7fd5ca249a4df5aba301beb23e2eb7, user.id=f16ca5f6955045a5811c25a8e6cc8055) {{(pid=81851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:56:49.916256 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73aaf4cb-0c9b-469a-8602-050b51be9854 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:56:49.917743 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73aaf4cb-0c9b-469a-8602-050b51be9854 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:56:49.929041 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 72/149] 10.222.0.62 () {56 vars in 1374 bytes} [Wed Aug 24 05:56:49 2022] GET /identity/v3/role_assignments?user.id=f16ca5f6955045a5811c25a8e6cc8055&scope.project.id=6d7fd5ca249a4df5aba301beb23e2eb7&role.id=580aa7dbb38d4cff8f9d1bb8983f99ef => generated 608 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:56:50.058663 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4b09b5-0913-4b00-88f0-2044bbfefd16 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:56:50.102067 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:56:50.186034 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4b09b5-0913-4b00-88f0-2044bbfefd16 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:56:50.225220 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048685a8-50b6-4ac8-bcbc-419b1b17cd18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:56:50.225857 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048685a8-50b6-4ac8-bcbc-419b1b17cd18 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:50.226131 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048685a8-50b6-4ac8-bcbc-419b1b17cd18 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:50.226395 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048685a8-50b6-4ac8-bcbc-419b1b17cd18 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:50.232320 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7b4a97-b752-4dae-87d4-7b5ff74ccc58 None None] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:50.232920 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7b4a97-b752-4dae-87d4-7b5ff74ccc58 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:50.233210 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7b4a97-b752-4dae-87d4-7b5ff74ccc58 None None] PATH_INFO: `/` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:50.234674 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 73/150] 10.222.0.62 () {54 vars in 927 bytes} [Wed Aug 24 05:56:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 05:56:50.242522 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048685a8-50b6-4ac8-bcbc-419b1b17cd18 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:56:50.243135 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8a64b9-44dc-4692-8e68-bf6b1af97859 None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:50.243416 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8a64b9-44dc-4692-8e68-bf6b1af97859 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:50.243688 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8a64b9-44dc-4692-8e68-bf6b1af97859 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:50.247326 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-048685a8-50b6-4ac8-bcbc-419b1b17cd18 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 05:56:50.247326 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:56:50.247326 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:56:50.247326 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:56:50.247326 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:56:50.247326 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:56:50.247326 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:56:50.247326 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:56:50.247326 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:56:50.247326 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:56:50.247326 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:56:50.247326 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:56:50.247326 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:56:50.247326 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:56:50.247326 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:56:50.247326 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:56:50.248179 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:56:50.248179 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:56:50.248179 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:56:50.248179 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:56:50.248179 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:56:50.248179 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:56:50.248179 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:56:50.248179 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:56:50.248179 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:56:50.248179 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:56:50.248179 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:56:50.248179 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 24 05:56:50.248179 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:56:50.248179 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:56:50.248179 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:56:50.248179 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:56:50.248976 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:56:50.248976 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:56:50.248976 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:56:50.248976 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:56:50.248976 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:56:50.248976 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:56:50.248976 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:56:50.248976 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:56:50.248976 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:56:50.248976 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:56:50.248976 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 05:56:50.248976 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 24 05:56:50.248976 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 78/151] 10.222.0.62 () {56 vars in 1096 bytes} [Wed Aug 24 05:56:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:56:50.256745 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048685a8-50b6-4ac8-bcbc-419b1b17cd18 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:56:50.277149 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-ba8a64b9-44dc-4692-8e68-bf6b1af97859 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:56:50.278345 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-ba8a64b9-44dc-4692-8e68-bf6b1af97859 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:56:50.401264 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:56:50.405195 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048685a8-50b6-4ac8-bcbc-419b1b17cd18 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:56:50.405654 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 74/152] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Aug 24 05:56:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Aug 24 05:56:50.406401 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8a64b9-44dc-4692-8e68-bf6b1af97859 None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:56:50.498199 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6174e1-0dbc-4332-a5c0-aff8c687613b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:56:50.498742 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6174e1-0dbc-4332-a5c0-aff8c687613b None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:50.499026 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6174e1-0dbc-4332-a5c0-aff8c687613b None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:50.499295 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6174e1-0dbc-4332-a5c0-aff8c687613b None admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:50.499966 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6174e1-0dbc-4332-a5c0-aff8c687613b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:56:50.500274 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6174e1-0dbc-4332-a5c0-aff8c687613b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:56:50.501693 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6174e1-0dbc-4332-a5c0-aff8c687613b None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:56:50.519962 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 79/153] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Aug 24 05:56:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:56:50.521232 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8a64b9-44dc-4692-8e68-bf6b1af97859 None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:56:50.525184 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6174e1-0dbc-4332-a5c0-aff8c687613b None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:56:50.541742 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:56:50.563657 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97556da6-70f0-47b8-a172-d9fdc045b510 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:56:50.564547 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97556da6-70f0-47b8-a172-d9fdc045b510 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:50.565072 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97556da6-70f0-47b8-a172-d9fdc045b510 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:50.565485 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97556da6-70f0-47b8-a172-d9fdc045b510 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:50.566727 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97556da6-70f0-47b8-a172-d9fdc045b510 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:56:50.568323 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97556da6-70f0-47b8-a172-d9fdc045b510 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:56:50.570696 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 75/154] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Aug 24 05:56:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:56:50.577329 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97556da6-70f0-47b8-a172-d9fdc045b510 None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:56:50.689128 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97556da6-70f0-47b8-a172-d9fdc045b510 None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:56:50.739284 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6174e1-0dbc-4332-a5c0-aff8c687613b None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:56:50.756288 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f671a96e-aa29-4d22-95fc-5fab8446d6cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:56:50.756288 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f671a96e-aa29-4d22-95fc-5fab8446d6cd None admin] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:50.756288 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f671a96e-aa29-4d22-95fc-5fab8446d6cd None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:50.756288 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f671a96e-aa29-4d22-95fc-5fab8446d6cd None admin] PATH_INFO: `/v3/users` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:50.757024 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f671a96e-aa29-4d22-95fc-5fab8446d6cd None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:56:50.758460 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f671a96e-aa29-4d22-95fc-5fab8446d6cd None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:56:50.794836 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf75ac7-2514-43be-9f2e-8dd8231d0a8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:56:50.795758 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf75ac7-2514-43be-9f2e-8dd8231d0a8a None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:50.796083 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf75ac7-2514-43be-9f2e-8dd8231d0a8a None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:50.796371 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf75ac7-2514-43be-9f2e-8dd8231d0a8a None admin] PATH_INFO: `/v3/users/8b2c3a376af245f2a0e347e2f08885a5` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:50.797664 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf75ac7-2514-43be-9f2e-8dd8231d0a8a None admin] RBAC: Authorizing `identity:get_user(user_id=8b2c3a376af245f2a0e347e2f08885a5)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:56:50.799330 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf75ac7-2514-43be-9f2e-8dd8231d0a8a None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:56:50.801021 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 80/155] 10.222.0.62 () {56 vars in 1177 bytes} [Wed Aug 24 05:56:50 2022] GET /identity/v3/users/8b2c3a376af245f2a0e347e2f08885a5 => generated 247 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:56:50.807297 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf75ac7-2514-43be-9f2e-8dd8231d0a8a None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:56:50.839319 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:56:50.880739 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 76/156] 10.222.0.62 () {60 vars in 1132 bytes} [Wed Aug 24 05:56:50 2022] POST /identity/v3/users => generated 283 bytes in 299 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 05:56:50.881841 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f671a96e-aa29-4d22-95fc-5fab8446d6cd None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:56:50.932229 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf75ac7-2514-43be-9f2e-8dd8231d0a8a None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:56:50.959395 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:56:50.974804 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd309e4-ad87-4139-b3c6-7875e5900368 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:56:50.975566 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd309e4-ad87-4139-b3c6-7875e5900368 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:50.976247 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd309e4-ad87-4139-b3c6-7875e5900368 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:50.976247 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd309e4-ad87-4139-b3c6-7875e5900368 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:51.049114 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f671a96e-aa29-4d22-95fc-5fab8446d6cd None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:56:51.060255 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd309e4-ad87-4139-b3c6-7875e5900368 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:56:51.062441 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd309e4-ad87-4139-b3c6-7875e5900368 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:56:51.096169 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b04b75a-9f79-440c-8b7f-a1f9c85c5b03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:56:51.097083 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b04b75a-9f79-440c-8b7f-a1f9c85c5b03 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:51.097460 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b04b75a-9f79-440c-8b7f-a1f9c85c5b03 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:51.097980 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b04b75a-9f79-440c-8b7f-a1f9c85c5b03 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:51.099070 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:56:51.111022 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b04b75a-9f79-440c-8b7f-a1f9c85c5b03 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:56:51.118431 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application [None req-6b04b75a-9f79-440c-8b7f-a1f9c85c5b03 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 05:56:51.118431 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:56:51.118431 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:56:51.118431 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:56:51.118431 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:56:51.118431 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:56:51.118431 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:56:51.118431 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:56:51.118431 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:56:51.118431 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:56:51.118431 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:56:51.118431 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:56:51.118431 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:56:51.118431 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:56:51.118431 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:56:51.118431 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:56:51.119441 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:56:51.119441 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:56:51.119441 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:56:51.119441 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:56:51.119441 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:56:51.119441 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:56:51.119441 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:56:51.119441 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:56:51.119441 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:56:51.119441 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:56:51.119441 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:56:51.119441 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application with Lock( Aug 24 05:56:51.119441 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:56:51.119441 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:56:51.119441 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:56:51.119441 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:56:51.120274 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:56:51.120274 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:56:51.120274 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:56:51.120274 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:56:51.120274 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:56:51.120274 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:56:51.120274 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:56:51.120274 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:56:51.120274 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:56:51.120274 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:56:51.120274 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 05:56:51.120274 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Aug 24 05:56:51.120274 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 77/157] 10.222.0.62 () {56 vars in 1114 bytes} [Wed Aug 24 05:56:50 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:56:51.121884 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302cd58f-e9b2-473d-83bb-67b3046b461d None None] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:51.121884 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302cd58f-e9b2-473d-83bb-67b3046b461d None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:51.121884 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302cd58f-e9b2-473d-83bb-67b3046b461d None None] PATH_INFO: `/` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:51.126312 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 78/158] 10.222.0.62 () {54 vars in 927 bytes} [Wed Aug 24 05:56:51 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:56:51.129969 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302cd58f-e9b2-473d-83bb-67b3046b461d None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:56:51.202157 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 81/159] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Aug 24 05:56:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 05:56:51.203650 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33c517c-66eb-4ee6-a1ed-afd6d7ff43f3 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:51.203944 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33c517c-66eb-4ee6-a1ed-afd6d7ff43f3 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:51.204218 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33c517c-66eb-4ee6-a1ed-afd6d7ff43f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:51.224944 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-a33c517c-66eb-4ee6-a1ed-afd6d7ff43f3 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:56:51.226140 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-a33c517c-66eb-4ee6-a1ed-afd6d7ff43f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:56:51.256549 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302cd58f-e9b2-473d-83bb-67b3046b461d None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:56:51.258511 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:56:51.296308 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5eb258-f203-4d26-8fb6-f7d52fad59ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:56:51.297501 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5eb258-f203-4d26-8fb6-f7d52fad59ac None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:51.297946 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5eb258-f203-4d26-8fb6-f7d52fad59ac None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:51.298227 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5eb258-f203-4d26-8fb6-f7d52fad59ac None admin] PATH_INFO: `/v3/roles` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:51.299069 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5eb258-f203-4d26-8fb6-f7d52fad59ac None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:56:51.300245 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5eb258-f203-4d26-8fb6-f7d52fad59ac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:56:51.302057 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5eb258-f203-4d26-8fb6-f7d52fad59ac None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:56:51.308231 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 82/160] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Aug 24 05:56:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 05:56:51.309762 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3199d72b-879c-4b69-a275-5bcd9288ec16 None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:51.310190 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3199d72b-879c-4b69-a275-5bcd9288ec16 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:51.310592 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3199d72b-879c-4b69-a275-5bcd9288ec16 None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:51.313043 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 83/161] 10.222.0.62 () {54 vars in 927 bytes} [Wed Aug 24 05:56:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:56:51.313943 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3199d72b-879c-4b69-a275-5bcd9288ec16 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:56:51.314354 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 79/162] 10.222.0.62 () {56 vars in 1111 bytes} [Wed Aug 24 05:56:51 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:56:51.321021 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5339363d-9119-44f6-bb82-840a106ecfbc None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:51.321373 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5339363d-9119-44f6-bb82-840a106ecfbc None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:51.321658 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5339363d-9119-44f6-bb82-840a106ecfbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:51.345131 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-5339363d-9119-44f6-bb82-840a106ecfbc None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:56:51.346594 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-5339363d-9119-44f6-bb82-840a106ecfbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:56:51.391164 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:56:51.431806 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 80/163] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Aug 24 05:56:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 05:56:51.433209 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5339363d-9119-44f6-bb82-840a106ecfbc None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:56:51.442252 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3199d72b-879c-4b69-a275-5bcd9288ec16 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:56:51.517346 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16414cfa-2ee6-4567-bf60-ca7f5aecc4c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:56:51.517923 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16414cfa-2ee6-4567-bf60-ca7f5aecc4c9 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:51.518800 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16414cfa-2ee6-4567-bf60-ca7f5aecc4c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:51.519083 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16414cfa-2ee6-4567-bf60-ca7f5aecc4c9 None admin] PATH_INFO: `/v3/projects/6d7fd5ca249a4df5aba301beb23e2eb7` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:51.520325 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16414cfa-2ee6-4567-bf60-ca7f5aecc4c9 None admin] RBAC: Authorizing `identity:get_project(project_id=6d7fd5ca249a4df5aba301beb23e2eb7)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:56:51.521720 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16414cfa-2ee6-4567-bf60-ca7f5aecc4c9 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:56:51.523831 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 84/164] 10.222.0.62 () {56 vars in 1186 bytes} [Wed Aug 24 05:56:51 2022] GET /identity/v3/projects/6d7fd5ca249a4df5aba301beb23e2eb7 => generated 298 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:56:51.530485 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16414cfa-2ee6-4567-bf60-ca7f5aecc4c9 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:56:51.542567 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5339363d-9119-44f6-bb82-840a106ecfbc None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:56:51.556925 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:56:51.577956 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5861c8ea-12dc-47ff-8a87-735554b37729 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:56:51.578675 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5861c8ea-12dc-47ff-8a87-735554b37729 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:51.579145 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5861c8ea-12dc-47ff-8a87-735554b37729 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:51.579509 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5861c8ea-12dc-47ff-8a87-735554b37729 None admin] PATH_INFO: `/v3/users/f16ca5f6955045a5811c25a8e6cc8055` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:51.581337 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5861c8ea-12dc-47ff-8a87-735554b37729 None admin] RBAC: Authorizing `identity:get_user(user_id=f16ca5f6955045a5811c25a8e6cc8055)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:56:51.588537 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5861c8ea-12dc-47ff-8a87-735554b37729 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:56:51.588537 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 81/165] 10.222.0.62 () {56 vars in 1177 bytes} [Wed Aug 24 05:56:51 2022] GET /identity/v3/users/f16ca5f6955045a5811c25a8e6cc8055 => generated 275 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:56:51.613575 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5861c8ea-12dc-47ff-8a87-735554b37729 None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:56:51.665604 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16414cfa-2ee6-4567-bf60-ca7f5aecc4c9 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:56:51.697675 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:56:51.716545 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5922d6aa-ddea-432b-85da-070cc34c0ff0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:56:51.716545 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5922d6aa-ddea-432b-85da-070cc34c0ff0 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:51.716545 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5922d6aa-ddea-432b-85da-070cc34c0ff0 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:51.716545 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5922d6aa-ddea-432b-85da-070cc34c0ff0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:51.720277 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5922d6aa-ddea-432b-85da-070cc34c0ff0 None admin] RBAC: Adding query filter params (role.id=49e47cd157d743f6905f1db07ec878a7, scope.project.id=6d7fd5ca249a4df5aba301beb23e2eb7, user.id=8b2c3a376af245f2a0e347e2f08885a5) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:56:51.720277 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5922d6aa-ddea-432b-85da-070cc34c0ff0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:56:51.720277 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5922d6aa-ddea-432b-85da-070cc34c0ff0 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:56:51.723655 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 85/166] 10.222.0.62 () {56 vars in 1374 bytes} [Wed Aug 24 05:56:51 2022] GET /identity/v3/role_assignments?user.id=8b2c3a376af245f2a0e347e2f08885a5&scope.project.id=6d7fd5ca249a4df5aba301beb23e2eb7&role.id=49e47cd157d743f6905f1db07ec878a7 => generated 608 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:56:51.729407 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5922d6aa-ddea-432b-85da-070cc34c0ff0 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:56:51.761050 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5861c8ea-12dc-47ff-8a87-735554b37729 None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:56:51.801759 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c39270d-4e80-403b-8096-ee2ee42856a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:56:51.802377 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c39270d-4e80-403b-8096-ee2ee42856a6 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:51.802659 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c39270d-4e80-403b-8096-ee2ee42856a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:51.802946 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c39270d-4e80-403b-8096-ee2ee42856a6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:51.840538 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:56:51.846213 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5922d6aa-ddea-432b-85da-070cc34c0ff0 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:56:51.886866 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c39270d-4e80-403b-8096-ee2ee42856a6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:56:51.888712 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c39270d-4e80-403b-8096-ee2ee42856a6 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:56:51.898209 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c67f12-c70e-4e21-bb7d-2297aaf347a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:56:51.898867 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c67f12-c70e-4e21-bb7d-2297aaf347a7 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:51.899149 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c67f12-c70e-4e21-bb7d-2297aaf347a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:51.899425 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c67f12-c70e-4e21-bb7d-2297aaf347a7 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:51.922622 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c67f12-c70e-4e21-bb7d-2297aaf347a7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:56:51.927764 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-d5c67f12-c70e-4e21-bb7d-2297aaf347a7 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 05:56:51.927764 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:56:51.927764 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:56:51.927764 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:56:51.927764 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:56:51.927764 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:56:51.927764 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:56:51.927764 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:56:51.927764 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:56:51.927764 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:56:51.927764 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:56:51.927764 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:56:51.927764 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:56:51.927764 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:56:51.927764 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:56:51.927764 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:56:51.928708 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:56:51.928708 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:56:51.928708 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:56:51.928708 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:56:51.928708 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:56:51.928708 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:56:51.928708 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:56:51.928708 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:56:51.928708 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:56:51.928708 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:56:51.928708 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:56:51.928708 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 24 05:56:51.928708 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:56:51.928708 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:56:51.928708 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:56:51.928708 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:56:51.929725 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:56:51.929725 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:56:51.929725 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:56:51.929725 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:56:51.929725 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:56:51.929725 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:56:51.929725 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:56:51.929725 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:56:51.929725 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:56:51.929725 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:56:51.929725 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 05:56:51.929725 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 24 05:56:51.937096 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 86/167] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Aug 24 05:56:51 2022] GET /identity/v3/roles/reader => generated 84 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:56:51.938012 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c67f12-c70e-4e21-bb7d-2297aaf347a7 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:56:52.004601 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 82/168] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Aug 24 05:56:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 05:56:52.006161 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c39270d-4e80-403b-8096-ee2ee42856a6 None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:56:52.025834 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:56:52.081855 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c67f12-c70e-4e21-bb7d-2297aaf347a7 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:56:52.092838 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:56:52.117518 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b56c21c-088c-444c-b97f-ab9cbb019ef4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:56:52.118279 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b56c21c-088c-444c-b97f-ab9cbb019ef4 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:52.118449 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b56c21c-088c-444c-b97f-ab9cbb019ef4 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:52.118691 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b56c21c-088c-444c-b97f-ab9cbb019ef4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:52.128796 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c39270d-4e80-403b-8096-ee2ee42856a6 None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:56:52.130954 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b56c21c-088c-444c-b97f-ab9cbb019ef4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:56:52.136242 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-0b56c21c-088c-444c-b97f-ab9cbb019ef4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 05:56:52.136242 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:56:52.136242 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:56:52.136242 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:56:52.136242 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:56:52.136242 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:56:52.136242 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:56:52.136242 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:56:52.136242 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:56:52.136242 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:56:52.136242 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:56:52.136242 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:56:52.136242 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:56:52.136242 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:56:52.136242 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:56:52.136242 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:56:52.137186 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:56:52.137186 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:56:52.137186 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:56:52.137186 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:56:52.137186 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:56:52.137186 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:56:52.137186 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:56:52.137186 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:56:52.137186 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:56:52.137186 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:56:52.137186 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:56:52.137186 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 24 05:56:52.137186 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:56:52.137186 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:56:52.137186 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:56:52.137186 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:56:52.138077 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:56:52.138077 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:56:52.138077 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:56:52.138077 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:56:52.138077 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:56:52.138077 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:56:52.138077 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:56:52.138077 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:56:52.138077 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:56:52.138077 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:56:52.138077 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 05:56:52.138077 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 24 05:56:52.138713 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 87/169] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Aug 24 05:56:51 2022] GET /identity/v3/roles/member => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:56:52.139157 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b56c21c-088c-444c-b97f-ab9cbb019ef4 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:56:52.169551 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42696c58-d9c6-4dd0-971f-2d5abe36229e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:56:52.170826 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42696c58-d9c6-4dd0-971f-2d5abe36229e None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:52.171337 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42696c58-d9c6-4dd0-971f-2d5abe36229e None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:52.171818 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42696c58-d9c6-4dd0-971f-2d5abe36229e None admin] PATH_INFO: `/v3/roles` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:52.172664 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42696c58-d9c6-4dd0-971f-2d5abe36229e None admin] RBAC: Adding query filter params (name=reader) {{(pid=81851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:56:52.173147 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42696c58-d9c6-4dd0-971f-2d5abe36229e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:56:52.175438 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42696c58-d9c6-4dd0-971f-2d5abe36229e None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:56:52.186042 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 83/170] 10.222.0.62 () {56 vars in 1101 bytes} [Wed Aug 24 05:56:52 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:56:52.187261 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42696c58-d9c6-4dd0-971f-2d5abe36229e None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:56:52.251014 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b56c21c-088c-444c-b97f-ab9cbb019ef4 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:56:52.263358 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:56:52.288216 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da3468f-608e-4aa7-89be-afe262c2eab4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:56:52.289187 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da3468f-608e-4aa7-89be-afe262c2eab4 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:52.289495 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da3468f-608e-4aa7-89be-afe262c2eab4 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:52.289773 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da3468f-608e-4aa7-89be-afe262c2eab4 None admin] PATH_INFO: `/v3/projects/6d7fd5ca249a4df5aba301beb23e2eb7` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:52.291989 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da3468f-608e-4aa7-89be-afe262c2eab4 None admin] RBAC: Authorizing `identity:get_project(project_id=6d7fd5ca249a4df5aba301beb23e2eb7)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:56:52.293081 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da3468f-608e-4aa7-89be-afe262c2eab4 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:56:52.301215 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 88/171] 10.222.0.62 () {56 vars in 1186 bytes} [Wed Aug 24 05:56:52 2022] GET /identity/v3/projects/6d7fd5ca249a4df5aba301beb23e2eb7 => generated 298 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:56:52.301962 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da3468f-608e-4aa7-89be-afe262c2eab4 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:56:52.306999 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42696c58-d9c6-4dd0-971f-2d5abe36229e None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:56:52.334852 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:56:52.350378 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62069213-3833-4196-bdc3-4288c7f3c2b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:56:52.350378 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62069213-3833-4196-bdc3-4288c7f3c2b1 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:52.350378 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62069213-3833-4196-bdc3-4288c7f3c2b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:52.350378 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62069213-3833-4196-bdc3-4288c7f3c2b1 None admin] PATH_INFO: `/v3/roles` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:52.351657 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62069213-3833-4196-bdc3-4288c7f3c2b1 None admin] RBAC: Adding query filter params (name=member) {{(pid=81851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:56:52.352179 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62069213-3833-4196-bdc3-4288c7f3c2b1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:56:52.353971 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62069213-3833-4196-bdc3-4288c7f3c2b1 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:56:52.365252 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 84/172] 10.222.0.62 () {56 vars in 1101 bytes} [Wed Aug 24 05:56:52 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:56:52.366672 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62069213-3833-4196-bdc3-4288c7f3c2b1 None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:56:52.419133 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da3468f-608e-4aa7-89be-afe262c2eab4 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:56:52.453802 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015aa7f2-2f72-4c2d-9f97-e7f5342d43f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:56:52.454437 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015aa7f2-2f72-4c2d-9f97-e7f5342d43f5 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:52.454716 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015aa7f2-2f72-4c2d-9f97-e7f5342d43f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:52.454986 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015aa7f2-2f72-4c2d-9f97-e7f5342d43f5 None admin] PATH_INFO: `/v3/users/d88bdbc9f42241d9bac9843db6e7ef02` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:52.456145 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015aa7f2-2f72-4c2d-9f97-e7f5342d43f5 None admin] RBAC: Authorizing `identity:get_user(user_id=d88bdbc9f42241d9bac9843db6e7ef02)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:56:52.457650 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015aa7f2-2f72-4c2d-9f97-e7f5342d43f5 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:56:52.459399 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 89/173] 10.222.0.62 () {56 vars in 1177 bytes} [Wed Aug 24 05:56:52 2022] GET /identity/v3/users/d88bdbc9f42241d9bac9843db6e7ef02 => generated 289 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:56:52.460003 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015aa7f2-2f72-4c2d-9f97-e7f5342d43f5 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:56:52.496532 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62069213-3833-4196-bdc3-4288c7f3c2b1 None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:56:52.507851 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:56:52.535330 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67432de-24c9-4c0f-a990-8facaf635dee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:56:52.536328 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67432de-24c9-4c0f-a990-8facaf635dee None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:52.536877 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67432de-24c9-4c0f-a990-8facaf635dee None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:52.537383 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67432de-24c9-4c0f-a990-8facaf635dee None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:52.538373 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67432de-24c9-4c0f-a990-8facaf635dee None admin] RBAC: Adding query filter params (role.id=ba3c2c3cb98c424f9ece19903a59c53c, scope.project.id=6d7fd5ca249a4df5aba301beb23e2eb7, user.id=f16ca5f6955045a5811c25a8e6cc8055) {{(pid=81851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:56:52.538896 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67432de-24c9-4c0f-a990-8facaf635dee None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:56:52.540613 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67432de-24c9-4c0f-a990-8facaf635dee None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:56:52.548244 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 85/174] 10.222.0.62 () {56 vars in 1374 bytes} [Wed Aug 24 05:56:52 2022] GET /identity/v3/role_assignments?user.id=f16ca5f6955045a5811c25a8e6cc8055&scope.project.id=6d7fd5ca249a4df5aba301beb23e2eb7&role.id=ba3c2c3cb98c424f9ece19903a59c53c => generated 608 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:56:52.550058 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67432de-24c9-4c0f-a990-8facaf635dee None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:56:52.624090 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015aa7f2-2f72-4c2d-9f97-e7f5342d43f5 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:56:52.644926 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:56:52.665808 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67432de-24c9-4c0f-a990-8facaf635dee None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:56:52.697284 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687c3b01-e048-4d40-a9d7-d6e6e1d06ee8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:56:52.697951 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687c3b01-e048-4d40-a9d7-d6e6e1d06ee8 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:52.698233 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687c3b01-e048-4d40-a9d7-d6e6e1d06ee8 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:52.698508 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687c3b01-e048-4d40-a9d7-d6e6e1d06ee8 None admin] PATH_INFO: `/v3/users/f16ca5f6955045a5811c25a8e6cc8055` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:52.699734 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687c3b01-e048-4d40-a9d7-d6e6e1d06ee8 None admin] RBAC: Authorizing `identity:get_user(user_id=f16ca5f6955045a5811c25a8e6cc8055)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:56:52.701247 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687c3b01-e048-4d40-a9d7-d6e6e1d06ee8 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:56:52.703165 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 90/175] 10.222.0.62 () {56 vars in 1177 bytes} [Wed Aug 24 05:56:52 2022] GET /identity/v3/users/f16ca5f6955045a5811c25a8e6cc8055 => generated 275 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:56:52.713585 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687c3b01-e048-4d40-a9d7-d6e6e1d06ee8 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:56:52.718029 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b622eb-f19d-4872-98f2-49ea462acdfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:56:52.719112 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b622eb-f19d-4872-98f2-49ea462acdfc None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:52.719652 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b622eb-f19d-4872-98f2-49ea462acdfc None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:52.720179 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b622eb-f19d-4872-98f2-49ea462acdfc None admin] PATH_INFO: `/v3/projects/6d7fd5ca249a4df5aba301beb23e2eb7` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:52.722308 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b622eb-f19d-4872-98f2-49ea462acdfc None admin] RBAC: Authorizing `identity:get_project(project_id=6d7fd5ca249a4df5aba301beb23e2eb7)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:56:52.724667 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b622eb-f19d-4872-98f2-49ea462acdfc None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:56:52.727346 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 86/176] 10.222.0.62 () {56 vars in 1186 bytes} [Wed Aug 24 05:56:52 2022] GET /identity/v3/projects/6d7fd5ca249a4df5aba301beb23e2eb7 => generated 298 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:56:52.733140 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b622eb-f19d-4872-98f2-49ea462acdfc None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:56:52.830479 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687c3b01-e048-4d40-a9d7-d6e6e1d06ee8 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:56:52.844866 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:56:52.868635 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37df9b1-28c5-493c-98d3-02efed33130b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:56:52.868635 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37df9b1-28c5-493c-98d3-02efed33130b None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:52.868635 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37df9b1-28c5-493c-98d3-02efed33130b None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:52.868635 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37df9b1-28c5-493c-98d3-02efed33130b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:52.938420 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37df9b1-28c5-493c-98d3-02efed33130b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:56:52.939940 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37df9b1-28c5-493c-98d3-02efed33130b None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:56:52.951236 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b622eb-f19d-4872-98f2-49ea462acdfc None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:56:52.981405 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:56:53.014555 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5dfe167-c27a-47ce-b47a-a757146c5125 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:56:53.015225 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dfe167-c27a-47ce-b47a-a757146c5125 None admin] REQUEST_METHOD: `PUT` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:53.015504 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dfe167-c27a-47ce-b47a-a757146c5125 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:53.015779 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dfe167-c27a-47ce-b47a-a757146c5125 None admin] PATH_INFO: `/v3/projects/6d7fd5ca249a4df5aba301beb23e2eb7/users/d88bdbc9f42241d9bac9843db6e7ef02/roles/e2f40474c6de4f5fadfa7efbe4087971` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:53.021863 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5dfe167-c27a-47ce-b47a-a757146c5125 None admin] RBAC: Authorizing `identity:create_grant(project_id=6d7fd5ca249a4df5aba301beb23e2eb7, user_id=d88bdbc9f42241d9bac9843db6e7ef02, role_id=e2f40474c6de4f5fadfa7efbe4087971)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:56:53.023329 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5dfe167-c27a-47ce-b47a-a757146c5125 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:56:53.043610 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 91/177] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Aug 24 05:56:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 05:56:53.049510 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37df9b1-28c5-493c-98d3-02efed33130b None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:56:53.068854 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:56:53.097807 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 87/178] 10.222.0.62 () {58 vars in 1439 bytes} [Wed Aug 24 05:56:52 2022] PUT /identity/v3/projects/6d7fd5ca249a4df5aba301beb23e2eb7/users/d88bdbc9f42241d9bac9843db6e7ef02/roles/e2f40474c6de4f5fadfa7efbe4087971 => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:56:53.157180 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37df9b1-28c5-493c-98d3-02efed33130b None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:56:53.195549 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756f372b-6321-4254-8998-c9024591abc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:56:53.196110 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756f372b-6321-4254-8998-c9024591abc3 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:53.196389 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756f372b-6321-4254-8998-c9024591abc3 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:53.196694 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756f372b-6321-4254-8998-c9024591abc3 None admin] PATH_INFO: `/v3/projects/d8715c59754a4b35b2e19f4759824cf7` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:53.197848 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756f372b-6321-4254-8998-c9024591abc3 None admin] RBAC: Authorizing `identity:get_project(project_id=d8715c59754a4b35b2e19f4759824cf7)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:56:53.199343 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756f372b-6321-4254-8998-c9024591abc3 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:56:53.201146 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 92/179] 10.222.0.62 () {56 vars in 1186 bytes} [Wed Aug 24 05:56:53 2022] GET /identity/v3/projects/d8715c59754a4b35b2e19f4759824cf7 => generated 312 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:56:53.207938 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5dfe167-c27a-47ce-b47a-a757146c5125 None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:56:53.332251 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5dfe167-c27a-47ce-b47a-a757146c5125 None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:56:53.370576 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4bceec-f896-43e1-8843-31bd627ecd28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:56:53.371184 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4bceec-f896-43e1-8843-31bd627ecd28 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:53.371463 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4bceec-f896-43e1-8843-31bd627ecd28 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:53.371795 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4bceec-f896-43e1-8843-31bd627ecd28 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:53.372543 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4bceec-f896-43e1-8843-31bd627ecd28 None admin] RBAC: Adding query filter params (role.id=580aa7dbb38d4cff8f9d1bb8983f99ef, scope.project.id=d8715c59754a4b35b2e19f4759824cf7, user.id=f16ca5f6955045a5811c25a8e6cc8055) {{(pid=81851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:56:53.372856 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4bceec-f896-43e1-8843-31bd627ecd28 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:56:53.374294 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4bceec-f896-43e1-8843-31bd627ecd28 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:56:53.380707 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 88/180] 10.222.0.62 () {56 vars in 1374 bytes} [Wed Aug 24 05:56:53 2022] GET /identity/v3/role_assignments?user.id=f16ca5f6955045a5811c25a8e6cc8055&scope.project.id=d8715c59754a4b35b2e19f4759824cf7&role.id=580aa7dbb38d4cff8f9d1bb8983f99ef => generated 608 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:56:53.843622 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b07ab6-bf2c-4164-ab4f-bfc0b13828ac None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:53.843958 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b07ab6-bf2c-4164-ab4f-bfc0b13828ac None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:53.844171 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b07ab6-bf2c-4164-ab4f-bfc0b13828ac None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:53.845847 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 93/181] 10.222.0.62 () {54 vars in 927 bytes} [Wed Aug 24 05:56:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:56:53.851265 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47051095-077c-451c-b1a2-79296a6c8587 None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:53.851557 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47051095-077c-451c-b1a2-79296a6c8587 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:53.851835 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47051095-077c-451c-b1a2-79296a6c8587 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:53.874952 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-47051095-077c-451c-b1a2-79296a6c8587 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:56:53.876143 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-47051095-077c-451c-b1a2-79296a6c8587 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:56:53.918721 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:56:53.951465 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 89/182] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Aug 24 05:56:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 05:56:54.129632 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3b06e5-bb18-47bb-83f3-0f860f6cd039 None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:54.130123 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3b06e5-bb18-47bb-83f3-0f860f6cd039 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:54.130123 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3b06e5-bb18-47bb-83f3-0f860f6cd039 None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:54.131721 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 94/183] 10.222.0.62 () {54 vars in 927 bytes} [Wed Aug 24 05:56:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:56:54.136886 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d4d875-b057-42d5-805e-eacc8da97779 None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:54.137688 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d4d875-b057-42d5-805e-eacc8da97779 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:54.138214 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d4d875-b057-42d5-805e-eacc8da97779 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:54.144832 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e9f402-7f1b-4335-8604-b5be741920f9 None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:54.145113 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e9f402-7f1b-4335-8604-b5be741920f9 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:54.145386 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e9f402-7f1b-4335-8604-b5be741920f9 None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:54.146793 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 95/184] 10.222.0.62 () {54 vars in 927 bytes} [Wed Aug 24 05:56:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 05:56:54.152609 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fd96a6-0118-4bbc-9c17-c23632f065e6 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:54.153139 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fd96a6-0118-4bbc-9c17-c23632f065e6 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:54.153617 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fd96a6-0118-4bbc-9c17-c23632f065e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:54.162536 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-81d4d875-b057-42d5-805e-eacc8da97779 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:56:54.163709 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-81d4d875-b057-42d5-805e-eacc8da97779 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:56:54.173016 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-51fd96a6-0118-4bbc-9c17-c23632f065e6 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:56:54.175030 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-51fd96a6-0118-4bbc-9c17-c23632f065e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:56:54.239301 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 90/185] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Aug 24 05:56:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 05:56:54.239301 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d4d875-b057-42d5-805e-eacc8da97779 None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:56:54.254509 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 96/186] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Aug 24 05:56:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 05:56:54.351239 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d4d875-b057-42d5-805e-eacc8da97779 None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:56:54.352773 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:56:54.386827 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fd3994-5a85-4ddc-8a62-974536d9ab98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:56:54.387378 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fd3994-5a85-4ddc-8a62-974536d9ab98 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:54.387664 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fd3994-5a85-4ddc-8a62-974536d9ab98 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:54.387939 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fd3994-5a85-4ddc-8a62-974536d9ab98 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:54.398957 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45fd3994-5a85-4ddc-8a62-974536d9ab98 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:56:54.403700 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application [None req-45fd3994-5a85-4ddc-8a62-974536d9ab98 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 05:56:54.403700 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:56:54.403700 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:56:54.403700 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:56:54.403700 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:56:54.403700 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:56:54.403700 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:56:54.403700 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:56:54.403700 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:56:54.403700 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:56:54.403700 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:56:54.403700 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:56:54.403700 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:56:54.403700 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:56:54.403700 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:56:54.403700 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:56:54.404607 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:56:54.404607 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:56:54.404607 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:56:54.404607 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:56:54.404607 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:56:54.404607 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:56:54.404607 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:56:54.404607 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:56:54.404607 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:56:54.404607 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:56:54.404607 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:56:54.404607 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application with Lock( Aug 24 05:56:54.404607 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:56:54.404607 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:56:54.404607 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:56:54.404607 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:56:54.405399 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:56:54.405399 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:56:54.405399 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:56:54.405399 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:56:54.405399 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:56:54.405399 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:56:54.405399 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:56:54.405399 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:56:54.405399 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:56:54.405399 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:56:54.405399 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 05:56:54.405399 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Aug 24 05:56:54.405399 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 91/187] 10.222.0.62 () {56 vars in 1096 bytes} [Wed Aug 24 05:56:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:56:54.411232 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fd96a6-0118-4bbc-9c17-c23632f065e6 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:56:54.426829 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:56:54.519976 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fd96a6-0118-4bbc-9c17-c23632f065e6 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:56:54.526078 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fd3994-5a85-4ddc-8a62-974536d9ab98 None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:56:54.558092 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cc7f21-af96-4ed7-979d-06d1f04d2973 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:56:54.558704 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cc7f21-af96-4ed7-979d-06d1f04d2973 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:54.558988 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cc7f21-af96-4ed7-979d-06d1f04d2973 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:54.559265 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cc7f21-af96-4ed7-979d-06d1f04d2973 None admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:54.559929 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05cc7f21-af96-4ed7-979d-06d1f04d2973 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:56:54.560235 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05cc7f21-af96-4ed7-979d-06d1f04d2973 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:56:54.561903 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05cc7f21-af96-4ed7-979d-06d1f04d2973 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:56:54.570743 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 97/188] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Aug 24 05:56:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:56:54.571476 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cc7f21-af96-4ed7-979d-06d1f04d2973 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:56:54.648237 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:56:54.654783 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fd3994-5a85-4ddc-8a62-974536d9ab98 None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:56:54.675632 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cc7f21-af96-4ed7-979d-06d1f04d2973 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:56:54.690470 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f899d99-0eff-478a-8181-6f2c9ad956e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:56:54.691099 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f899d99-0eff-478a-8181-6f2c9ad956e8 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:54.691379 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f899d99-0eff-478a-8181-6f2c9ad956e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:54.691647 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f899d99-0eff-478a-8181-6f2c9ad956e8 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:54.702333 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f899d99-0eff-478a-8181-6f2c9ad956e8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:56:54.707498 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application [None req-5f899d99-0eff-478a-8181-6f2c9ad956e8 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 05:56:54.707498 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:56:54.707498 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:56:54.707498 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:56:54.707498 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:56:54.707498 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:56:54.707498 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:56:54.707498 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:56:54.707498 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:56:54.707498 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:56:54.707498 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:56:54.707498 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:56:54.707498 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:56:54.707498 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:56:54.707498 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:56:54.707498 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:56:54.708515 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:56:54.708515 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:56:54.708515 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:56:54.708515 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:56:54.708515 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:56:54.708515 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:56:54.708515 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:56:54.708515 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:56:54.708515 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:56:54.708515 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:56:54.708515 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:56:54.708515 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application with Lock( Aug 24 05:56:54.708515 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:56:54.708515 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:56:54.708515 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:56:54.708515 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:56:54.710370 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:56:54.710370 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:56:54.710370 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:56:54.710370 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:56:54.710370 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:56:54.710370 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:56:54.710370 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:56:54.710370 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:56:54.710370 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:56:54.710370 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:56:54.710370 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 05:56:54.710370 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Aug 24 05:56:54.710370 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 92/189] 10.222.0.62 () {56 vars in 1114 bytes} [Wed Aug 24 05:56:54 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 05:56:54.714162 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f899d99-0eff-478a-8181-6f2c9ad956e8 None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:56:54.719299 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74097e2f-0093-455b-a3ec-0fed94df9b16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:56:54.719935 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74097e2f-0093-455b-a3ec-0fed94df9b16 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:54.720211 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74097e2f-0093-455b-a3ec-0fed94df9b16 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:54.720479 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74097e2f-0093-455b-a3ec-0fed94df9b16 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:54.722089 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74097e2f-0093-455b-a3ec-0fed94df9b16 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:56:54.724071 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74097e2f-0093-455b-a3ec-0fed94df9b16 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:56:54.726044 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 98/190] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Aug 24 05:56:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:56:54.729322 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74097e2f-0093-455b-a3ec-0fed94df9b16 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:56:54.747539 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:56:54.833425 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74097e2f-0093-455b-a3ec-0fed94df9b16 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:56:54.839106 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f899d99-0eff-478a-8181-6f2c9ad956e8 None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:56:54.869604 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7015c16-1d72-46fd-b8e0-1ffc39440927 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:56:54.870217 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7015c16-1d72-46fd-b8e0-1ffc39440927 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:54.870503 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7015c16-1d72-46fd-b8e0-1ffc39440927 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:54.870776 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7015c16-1d72-46fd-b8e0-1ffc39440927 None admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:54.871433 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7015c16-1d72-46fd-b8e0-1ffc39440927 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:56:54.871758 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7015c16-1d72-46fd-b8e0-1ffc39440927 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:56:54.873146 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7015c16-1d72-46fd-b8e0-1ffc39440927 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:56:54.878081 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a387574f-75a5-4ab3-85e2-b5e68bd03116 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:56:54.879239 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a387574f-75a5-4ab3-85e2-b5e68bd03116 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:54.879783 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a387574f-75a5-4ab3-85e2-b5e68bd03116 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:54.880066 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a387574f-75a5-4ab3-85e2-b5e68bd03116 None admin] PATH_INFO: `/v3/users/26afd7e3fe114d88a1d0e06e544c91eb` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:54.881453 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 99/191] 10.222.0.62 () {56 vars in 1111 bytes} [Wed Aug 24 05:56:54 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:56:54.886694 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7015c16-1d72-46fd-b8e0-1ffc39440927 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:56:54.891065 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:56:54.896742 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a387574f-75a5-4ab3-85e2-b5e68bd03116 None admin] RBAC: Authorizing `identity:get_user(user_id=26afd7e3fe114d88a1d0e06e544c91eb)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:56:54.898452 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a387574f-75a5-4ab3-85e2-b5e68bd03116 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:56:54.904461 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 93/192] 10.222.0.62 () {56 vars in 1177 bytes} [Wed Aug 24 05:56:54 2022] GET /identity/v3/users/26afd7e3fe114d88a1d0e06e544c91eb => generated 283 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:56:54.904461 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a387574f-75a5-4ab3-85e2-b5e68bd03116 None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:56:54.986736 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:56:54.999921 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7015c16-1d72-46fd-b8e0-1ffc39440927 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:56:55.006050 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a387574f-75a5-4ab3-85e2-b5e68bd03116 None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:56:55.038474 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196ceec1-c28d-4b75-b084-05c373d935b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:56:55.039086 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196ceec1-c28d-4b75-b084-05c373d935b4 None admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:55.039365 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196ceec1-c28d-4b75-b084-05c373d935b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:55.039636 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196ceec1-c28d-4b75-b084-05c373d935b4 None admin] PATH_INFO: `/v3/users` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:55.040279 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196ceec1-c28d-4b75-b084-05c373d935b4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:56:55.041776 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196ceec1-c28d-4b75-b084-05c373d935b4 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:56:55.045531 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a84229-de9c-445c-8e22-af031efbf31b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:56:55.046365 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a84229-de9c-445c-8e22-af031efbf31b None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:55.046812 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a84229-de9c-445c-8e22-af031efbf31b None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:55.047250 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a84229-de9c-445c-8e22-af031efbf31b None admin] PATH_INFO: `/v3/users/26afd7e3fe114d88a1d0e06e544c91eb` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:55.048539 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a84229-de9c-445c-8e22-af031efbf31b None admin] RBAC: Authorizing `identity:get_user(user_id=26afd7e3fe114d88a1d0e06e544c91eb)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:56:55.050724 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a84229-de9c-445c-8e22-af031efbf31b None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:56:55.052506 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 94/193] 10.222.0.62 () {56 vars in 1177 bytes} [Wed Aug 24 05:56:54 2022] GET /identity/v3/users/26afd7e3fe114d88a1d0e06e544c91eb => generated 283 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:56:55.053418 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a84229-de9c-445c-8e22-af031efbf31b None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:56:55.154438 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a84229-de9c-445c-8e22-af031efbf31b None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:56:55.188842 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4563ceb-fb83-4c2b-8d06-0463c43000cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:56:55.189733 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4563ceb-fb83-4c2b-8d06-0463c43000cf None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:55.196663 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4563ceb-fb83-4c2b-8d06-0463c43000cf None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:55.196846 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 100/194] 10.222.0.62 () {60 vars in 1132 bytes} [Wed Aug 24 05:56:54 2022] POST /identity/v3/users => generated 297 bytes in 308 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:56:55.197288 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4563ceb-fb83-4c2b-8d06-0463c43000cf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:55.197377 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196ceec1-c28d-4b75-b084-05c373d935b4 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:56:55.219602 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:56:55.281735 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4563ceb-fb83-4c2b-8d06-0463c43000cf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:56:55.284338 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4563ceb-fb83-4c2b-8d06-0463c43000cf None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:56:55.292689 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:56:55.349992 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196ceec1-c28d-4b75-b084-05c373d935b4 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:56:55.391126 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf08d89-50b8-469b-9a78-4f32862f54ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:56:55.391740 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf08d89-50b8-469b-9a78-4f32862f54ab None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:55.392020 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf08d89-50b8-469b-9a78-4f32862f54ab None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:55.392278 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf08d89-50b8-469b-9a78-4f32862f54ab None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:55.409198 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 95/195] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Aug 24 05:56:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 05:56:55.410226 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fca2d5a-4a82-44b7-a770-e17a6275056f None None] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:55.410642 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fca2d5a-4a82-44b7-a770-e17a6275056f None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:55.410769 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fca2d5a-4a82-44b7-a770-e17a6275056f None None] PATH_INFO: `/` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:55.412030 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 96/196] 10.222.0.62 () {54 vars in 927 bytes} [Wed Aug 24 05:56:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:56:55.414501 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fca2d5a-4a82-44b7-a770-e17a6275056f None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:56:55.476974 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:56:55.482234 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf08d89-50b8-469b-9a78-4f32862f54ab None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:56:55.483742 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf08d89-50b8-469b-9a78-4f32862f54ab None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:56:55.541701 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fca2d5a-4a82-44b7-a770-e17a6275056f None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:56:55.552667 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:56:55.587425 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3c716d-38de-4f6e-adda-6f78db44414d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:56:55.588071 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3c716d-38de-4f6e-adda-6f78db44414d None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:55.588354 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3c716d-38de-4f6e-adda-6f78db44414d None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:55.588662 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3c716d-38de-4f6e-adda-6f78db44414d None admin] PATH_INFO: `/v3/projects/9391bbe626b44799942ef25c41bc33ad` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:55.590336 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3c716d-38de-4f6e-adda-6f78db44414d None admin] RBAC: Authorizing `identity:get_project(project_id=9391bbe626b44799942ef25c41bc33ad)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:56:55.592300 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3c716d-38de-4f6e-adda-6f78db44414d None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:56:55.599280 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 97/197] 10.222.0.62 () {56 vars in 1186 bytes} [Wed Aug 24 05:56:55 2022] GET /identity/v3/projects/9391bbe626b44799942ef25c41bc33ad => generated 302 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:56:55.599280 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe46202-98bc-4400-91f2-5cb56d6aacb9 None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:55.599280 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe46202-98bc-4400-91f2-5cb56d6aacb9 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:55.599280 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe46202-98bc-4400-91f2-5cb56d6aacb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:55.603329 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 101/198] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Aug 24 05:56:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 05:56:55.604433 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf08d89-50b8-469b-9a78-4f32862f54ab None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:56:55.633291 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-0fe46202-98bc-4400-91f2-5cb56d6aacb9 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:56:55.635127 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-0fe46202-98bc-4400-91f2-5cb56d6aacb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:56:55.720215 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf08d89-50b8-469b-9a78-4f32862f54ab None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:56:55.735545 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 98/199] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Aug 24 05:56:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 05:56:55.736715 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe46202-98bc-4400-91f2-5cb56d6aacb9 None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:56:55.756829 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1504c0-76f6-4fff-8299-d01e303a5ba2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:56:55.757786 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1504c0-76f6-4fff-8299-d01e303a5ba2 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:55.758203 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1504c0-76f6-4fff-8299-d01e303a5ba2 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:55.758610 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1504c0-76f6-4fff-8299-d01e303a5ba2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:55.759962 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1504c0-76f6-4fff-8299-d01e303a5ba2 None admin] RBAC: Adding query filter params (role.id=49e47cd157d743f6905f1db07ec878a7, scope.project.id=9391bbe626b44799942ef25c41bc33ad, user.id=26afd7e3fe114d88a1d0e06e544c91eb) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:56:55.759962 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1504c0-76f6-4fff-8299-d01e303a5ba2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:56:55.761641 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1504c0-76f6-4fff-8299-d01e303a5ba2 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:56:55.768708 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 102/200] 10.222.0.62 () {56 vars in 1374 bytes} [Wed Aug 24 05:56:55 2022] GET /identity/v3/role_assignments?user.id=26afd7e3fe114d88a1d0e06e544c91eb&scope.project.id=9391bbe626b44799942ef25c41bc33ad&role.id=49e47cd157d743f6905f1db07ec878a7 => generated 259 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:56:55.796904 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:56:55.860901 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe46202-98bc-4400-91f2-5cb56d6aacb9 None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:56:55.900116 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4939d1cc-3dbe-4b19-9024-6b445cffcc4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:56:55.900786 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4939d1cc-3dbe-4b19-9024-6b445cffcc4a None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:55.901121 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4939d1cc-3dbe-4b19-9024-6b445cffcc4a None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:55.901396 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4939d1cc-3dbe-4b19-9024-6b445cffcc4a None admin] PATH_INFO: `/v3/projects/9391bbe626b44799942ef25c41bc33ad` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:55.902557 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4939d1cc-3dbe-4b19-9024-6b445cffcc4a None admin] RBAC: Authorizing `identity:get_project(project_id=9391bbe626b44799942ef25c41bc33ad)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:56:55.904052 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4939d1cc-3dbe-4b19-9024-6b445cffcc4a None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:56:55.906101 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 99/201] 10.222.0.62 () {56 vars in 1186 bytes} [Wed Aug 24 05:56:55 2022] GET /identity/v3/projects/9391bbe626b44799942ef25c41bc33ad => generated 302 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:56:55.912651 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1504c0-76f6-4fff-8299-d01e303a5ba2 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:56:55.969583 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:56:56.068614 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1504c0-76f6-4fff-8299-d01e303a5ba2 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:56:56.115014 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda55337-2a13-4091-9231-5b41288f3128 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:56:56.115866 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda55337-2a13-4091-9231-5b41288f3128 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:56.116286 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda55337-2a13-4091-9231-5b41288f3128 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:56.116722 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda55337-2a13-4091-9231-5b41288f3128 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:56.117580 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda55337-2a13-4091-9231-5b41288f3128 None admin] RBAC: Adding query filter params (role.id=ba3c2c3cb98c424f9ece19903a59c53c, scope.project.id=9391bbe626b44799942ef25c41bc33ad, user.id=26afd7e3fe114d88a1d0e06e544c91eb) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:56:56.118109 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda55337-2a13-4091-9231-5b41288f3128 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:56:56.119689 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda55337-2a13-4091-9231-5b41288f3128 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:56:56.127119 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 103/202] 10.222.0.62 () {56 vars in 1374 bytes} [Wed Aug 24 05:56:55 2022] GET /identity/v3/role_assignments?user.id=26afd7e3fe114d88a1d0e06e544c91eb&scope.project.id=9391bbe626b44799942ef25c41bc33ad&role.id=ba3c2c3cb98c424f9ece19903a59c53c => generated 259 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:56:56.147359 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4939d1cc-3dbe-4b19-9024-6b445cffcc4a None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:56:56.284620 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4939d1cc-3dbe-4b19-9024-6b445cffcc4a None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:56:56.304888 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:56:56.320015 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d083e33-51ae-4443-ba56-8668285be078 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:56:56.320742 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d083e33-51ae-4443-ba56-8668285be078 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:56.321120 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d083e33-51ae-4443-ba56-8668285be078 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:56.321397 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d083e33-51ae-4443-ba56-8668285be078 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:56.334975 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d083e33-51ae-4443-ba56-8668285be078 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:56:56.340360 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application [None req-4d083e33-51ae-4443-ba56-8668285be078 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 05:56:56.340360 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:56:56.340360 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:56:56.340360 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:56:56.340360 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:56:56.340360 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:56:56.340360 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:56:56.340360 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:56:56.340360 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:56:56.340360 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:56:56.340360 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:56:56.340360 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:56:56.340360 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:56:56.340360 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:56:56.340360 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:56:56.340360 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:56:56.341337 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:56:56.341337 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:56:56.341337 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:56:56.341337 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:56:56.341337 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:56:56.341337 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:56:56.341337 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:56:56.341337 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:56:56.341337 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:56:56.341337 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:56:56.341337 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:56:56.341337 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application with Lock( Aug 24 05:56:56.341337 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:56:56.341337 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:56:56.341337 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:56:56.341337 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:56:56.342193 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:56:56.342193 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:56:56.342193 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:56:56.342193 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:56:56.342193 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:56:56.342193 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:56:56.342193 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:56:56.342193 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:56:56.342193 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:56:56.342193 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:56:56.342193 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 05:56:56.342193 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Aug 24 05:56:56.345432 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 100/203] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Aug 24 05:56:56 2022] GET /identity/v3/roles/reader => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:56:56.351224 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda55337-2a13-4091-9231-5b41288f3128 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:56:56.473616 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:56:56.488610 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda55337-2a13-4091-9231-5b41288f3128 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:56:56.536612 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61ae9f7-e7a4-4f9a-9c98-eba0c8b0003e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:56:56.537138 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61ae9f7-e7a4-4f9a-9c98-eba0c8b0003e None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:56.537414 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61ae9f7-e7a4-4f9a-9c98-eba0c8b0003e None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:56.537680 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61ae9f7-e7a4-4f9a-9c98-eba0c8b0003e None admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:56.538344 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61ae9f7-e7a4-4f9a-9c98-eba0c8b0003e None admin] RBAC: Adding query filter params (name=reader) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:56:56.538649 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61ae9f7-e7a4-4f9a-9c98-eba0c8b0003e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:56:56.540023 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61ae9f7-e7a4-4f9a-9c98-eba0c8b0003e None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:56:56.552821 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 104/204] 10.222.0.62 () {56 vars in 1101 bytes} [Wed Aug 24 05:56:56 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:56:56.558514 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d083e33-51ae-4443-ba56-8668285be078 None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:56:56.691858 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d083e33-51ae-4443-ba56-8668285be078 None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:56:56.742881 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3cbe17-54c6-4d5c-8c54-12cf3ef5a294 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:56:56.743519 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3cbe17-54c6-4d5c-8c54-12cf3ef5a294 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:56.743802 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3cbe17-54c6-4d5c-8c54-12cf3ef5a294 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:56.744075 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3cbe17-54c6-4d5c-8c54-12cf3ef5a294 None admin] PATH_INFO: `/v3/users/d88bdbc9f42241d9bac9843db6e7ef02` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:56.747460 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3cbe17-54c6-4d5c-8c54-12cf3ef5a294 None admin] RBAC: Authorizing `identity:get_user(user_id=d88bdbc9f42241d9bac9843db6e7ef02)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:56:56.748970 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3cbe17-54c6-4d5c-8c54-12cf3ef5a294 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:56:56.750932 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 101/205] 10.222.0.62 () {56 vars in 1177 bytes} [Wed Aug 24 05:56:56 2022] GET /identity/v3/users/d88bdbc9f42241d9bac9843db6e7ef02 => generated 289 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:56:56.757058 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61ae9f7-e7a4-4f9a-9c98-eba0c8b0003e None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:56:56.926812 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61ae9f7-e7a4-4f9a-9c98-eba0c8b0003e None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:56:56.969134 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4342e815-4386-4ef4-8a3f-58e7cf9f4e29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:56:56.969746 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4342e815-4386-4ef4-8a3f-58e7cf9f4e29 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:56.970029 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4342e815-4386-4ef4-8a3f-58e7cf9f4e29 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:56.970298 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4342e815-4386-4ef4-8a3f-58e7cf9f4e29 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:56.996607 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:56:57.055486 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4342e815-4386-4ef4-8a3f-58e7cf9f4e29 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:56:57.056988 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4342e815-4386-4ef4-8a3f-58e7cf9f4e29 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:56:57.178984 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 105/206] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Aug 24 05:56:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 05:56:57.184778 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3cbe17-54c6-4d5c-8c54-12cf3ef5a294 None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:56:57.243938 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:56:57.316211 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3cbe17-54c6-4d5c-8c54-12cf3ef5a294 None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:56:57.363094 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd85769-f6f1-47bd-aa7f-b26fa141354b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:56:57.363745 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd85769-f6f1-47bd-aa7f-b26fa141354b None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:57.364024 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd85769-f6f1-47bd-aa7f-b26fa141354b None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:57.364294 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd85769-f6f1-47bd-aa7f-b26fa141354b None admin] PATH_INFO: `/v3/projects/6d7fd5ca249a4df5aba301beb23e2eb7` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:57.365446 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd85769-f6f1-47bd-aa7f-b26fa141354b None admin] RBAC: Authorizing `identity:get_project(project_id=6d7fd5ca249a4df5aba301beb23e2eb7)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:56:57.366895 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd85769-f6f1-47bd-aa7f-b26fa141354b None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:56:57.368544 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 102/207] 10.222.0.62 () {56 vars in 1186 bytes} [Wed Aug 24 05:56:57 2022] GET /identity/v3/projects/6d7fd5ca249a4df5aba301beb23e2eb7 => generated 298 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:56:57.374610 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4342e815-4386-4ef4-8a3f-58e7cf9f4e29 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:56:57.438683 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:56:57.499599 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4342e815-4386-4ef4-8a3f-58e7cf9f4e29 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:56:57.520564 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cd7513-bc19-4fc9-9f2b-fab52854594a None None] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:57.520841 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cd7513-bc19-4fc9-9f2b-fab52854594a None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:57.521184 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cd7513-bc19-4fc9-9f2b-fab52854594a None None] PATH_INFO: `/` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:57.522782 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 103/208] 10.222.0.62 () {54 vars in 927 bytes} [Wed Aug 24 05:56:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 05:56:57.527783 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3cf1f0-f909-4993-8f42-d596740c8758 None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:57.528061 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3cf1f0-f909-4993-8f42-d596740c8758 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:57.528327 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3cf1f0-f909-4993-8f42-d596740c8758 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:57.546008 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b37c24-e553-4588-b9ad-2e25ced2b135 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:56:57.546623 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b37c24-e553-4588-b9ad-2e25ced2b135 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:57.546899 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b37c24-e553-4588-b9ad-2e25ced2b135 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:57.547168 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b37c24-e553-4588-b9ad-2e25ced2b135 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:57.547898 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b37c24-e553-4588-b9ad-2e25ced2b135 None admin] RBAC: Adding query filter params (role.id=e2f40474c6de4f5fadfa7efbe4087971, scope.project.id=6d7fd5ca249a4df5aba301beb23e2eb7, user.id=d88bdbc9f42241d9bac9843db6e7ef02) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:56:57.548202 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b37c24-e553-4588-b9ad-2e25ced2b135 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:56:57.550267 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b37c24-e553-4588-b9ad-2e25ced2b135 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:56:57.560131 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-5f3cf1f0-f909-4993-8f42-d596740c8758 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:56:57.561345 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-5f3cf1f0-f909-4993-8f42-d596740c8758 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:56:57.565859 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 106/209] 10.222.0.62 () {56 vars in 1374 bytes} [Wed Aug 24 05:56:57 2022] GET /identity/v3/role_assignments?user.id=d88bdbc9f42241d9bac9843db6e7ef02&scope.project.id=6d7fd5ca249a4df5aba301beb23e2eb7&role.id=e2f40474c6de4f5fadfa7efbe4087971 => generated 608 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:56:57.593135 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468a464d-1568-446d-96ae-f21cf127b4d8 None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:57.593440 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468a464d-1568-446d-96ae-f21cf127b4d8 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:57.593683 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468a464d-1568-446d-96ae-f21cf127b4d8 None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:57.595022 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 107/210] 10.222.0.62 () {54 vars in 927 bytes} [Wed Aug 24 05:56:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 05:56:57.600560 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a465dd-cc61-462e-83cd-a687cd0178aa None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:57.600852 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a465dd-cc61-462e-83cd-a687cd0178aa None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:57.601179 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a465dd-cc61-462e-83cd-a687cd0178aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:57.630877 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-72a465dd-cc61-462e-83cd-a687cd0178aa None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:56:57.632096 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-72a465dd-cc61-462e-83cd-a687cd0178aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:56:57.659470 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 104/211] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Aug 24 05:56:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 05:56:57.758659 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 108/212] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Aug 24 05:56:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 05:56:57.974524 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3cf1f0-f909-4993-8f42-d596740c8758 None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:56:58.018073 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:56:58.041527 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a465dd-cc61-462e-83cd-a687cd0178aa None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:56:58.089302 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3cf1f0-f909-4993-8f42-d596740c8758 None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:56:58.101565 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:56:58.125588 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e29e606-8947-4310-937b-ac3573587d51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:56:58.126592 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e29e606-8947-4310-937b-ac3573587d51 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:58.127096 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e29e606-8947-4310-937b-ac3573587d51 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:58.127590 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e29e606-8947-4310-937b-ac3573587d51 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:58.128986 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e29e606-8947-4310-937b-ac3573587d51 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:56:58.130707 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e29e606-8947-4310-937b-ac3573587d51 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:56:58.132584 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 105/213] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Aug 24 05:56:57 2022] GET /identity/v3/domains/default => generated 199 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:56:58.138426 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e29e606-8947-4310-937b-ac3573587d51 None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:56:58.170105 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a465dd-cc61-462e-83cd-a687cd0178aa None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:56:58.207668 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa854e7c-fdf6-4c3e-8c61-8cfb76d40987 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:56:58.208314 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa854e7c-fdf6-4c3e-8c61-8cfb76d40987 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:58.208648 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa854e7c-fdf6-4c3e-8c61-8cfb76d40987 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:58.208928 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa854e7c-fdf6-4c3e-8c61-8cfb76d40987 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:58.219794 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa854e7c-fdf6-4c3e-8c61-8cfb76d40987 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:56:58.225275 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-aa854e7c-fdf6-4c3e-8c61-8cfb76d40987 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 05:56:58.225275 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:56:58.225275 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:56:58.225275 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:56:58.225275 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:56:58.225275 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:56:58.225275 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:56:58.225275 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:56:58.225275 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:56:58.225275 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:56:58.225275 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:56:58.225275 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:56:58.225275 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:56:58.225275 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:56:58.225275 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:56:58.225275 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:56:58.226143 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:56:58.226143 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:56:58.226143 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:56:58.226143 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:56:58.226143 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:56:58.226143 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:56:58.226143 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:56:58.226143 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:56:58.226143 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:56:58.226143 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:56:58.226143 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:56:58.226143 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 24 05:56:58.226143 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:56:58.226143 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:56:58.226143 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:56:58.226143 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:56:58.226923 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:56:58.226923 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:56:58.226923 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:56:58.226923 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:56:58.226923 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:56:58.226923 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:56:58.226923 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:56:58.226923 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:56:58.226923 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:56:58.226923 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:56:58.226923 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 05:56:58.226923 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 24 05:56:58.226923 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 109/214] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Aug 24 05:56:58 2022] GET /identity/v3/roles/member => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 05:56:58.233384 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec6a56e-9822-4ffd-b4ba-f8cf6caa1da4 None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:58.233670 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec6a56e-9822-4ffd-b4ba-f8cf6caa1da4 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:58.233945 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec6a56e-9822-4ffd-b4ba-f8cf6caa1da4 None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:58.235284 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 110/215] 10.222.0.62 () {54 vars in 927 bytes} [Wed Aug 24 05:56:58 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:56:58.235913 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec6a56e-9822-4ffd-b4ba-f8cf6caa1da4 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:56:58.256308 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e29e606-8947-4310-937b-ac3573587d51 None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:56:58.265547 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:56:58.291836 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7a3d01-157b-4006-ac39-ef4c157e36b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:56:58.292441 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7a3d01-157b-4006-ac39-ef4c157e36b4 None admin] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:58.292746 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7a3d01-157b-4006-ac39-ef4c157e36b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:58.293020 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7a3d01-157b-4006-ac39-ef4c157e36b4 None admin] PATH_INFO: `/v3/users` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:58.293766 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7a3d01-157b-4006-ac39-ef4c157e36b4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:56:58.295566 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7a3d01-157b-4006-ac39-ef4c157e36b4 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:56:58.344357 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec6a56e-9822-4ffd-b4ba-f8cf6caa1da4 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:56:58.347337 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:56:58.377608 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3121472-84cc-40f6-8b3a-f48b6d3e1e0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:56:58.378294 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3121472-84cc-40f6-8b3a-f48b6d3e1e0e None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:58.378524 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3121472-84cc-40f6-8b3a-f48b6d3e1e0e None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:58.378733 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3121472-84cc-40f6-8b3a-f48b6d3e1e0e None admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:58.379385 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3121472-84cc-40f6-8b3a-f48b6d3e1e0e None admin] RBAC: Adding query filter params (name=member) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:56:58.379682 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3121472-84cc-40f6-8b3a-f48b6d3e1e0e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:56:58.381099 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3121472-84cc-40f6-8b3a-f48b6d3e1e0e None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:56:58.388929 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 111/216] 10.222.0.62 () {56 vars in 1101 bytes} [Wed Aug 24 05:56:58 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:56:58.391141 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b6098f-8c27-4f33-8a41-841357d8252f None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:58.391468 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b6098f-8c27-4f33-8a41-841357d8252f None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:58.391744 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b6098f-8c27-4f33-8a41-841357d8252f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:58.409580 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-b8b6098f-8c27-4f33-8a41-841357d8252f None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:56:58.410736 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-b8b6098f-8c27-4f33-8a41-841357d8252f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:56:58.429462 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 106/217] 10.222.0.62 () {60 vars in 1132 bytes} [Wed Aug 24 05:56:58 2022] POST /identity/v3/users => generated 297 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:56:58.429462 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7a3d01-157b-4006-ac39-ef4c157e36b4 None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:56:58.493707 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 112/218] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Aug 24 05:56:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 05:56:58.559575 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7a3d01-157b-4006-ac39-ef4c157e36b4 None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:56:58.597236 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56b40c3-09dd-4d1f-bec7-ad26c528352d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:56:58.597891 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56b40c3-09dd-4d1f-bec7-ad26c528352d None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:58.598159 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56b40c3-09dd-4d1f-bec7-ad26c528352d None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:58.598432 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56b40c3-09dd-4d1f-bec7-ad26c528352d None admin] PATH_INFO: `/v3/users/6e01f2a8b34743a9b2a7cc1d67be9890` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:58.612440 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56b40c3-09dd-4d1f-bec7-ad26c528352d None admin] RBAC: Authorizing `identity:get_user(user_id=6e01f2a8b34743a9b2a7cc1d67be9890)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:56:58.613940 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56b40c3-09dd-4d1f-bec7-ad26c528352d None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:56:58.615842 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 107/219] 10.222.0.62 () {56 vars in 1177 bytes} [Wed Aug 24 05:56:58 2022] GET /identity/v3/users/6e01f2a8b34743a9b2a7cc1d67be9890 => generated 297 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:56:58.621399 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b6098f-8c27-4f33-8a41-841357d8252f None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:56:58.721945 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:56:58.727844 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b6098f-8c27-4f33-8a41-841357d8252f None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:56:58.763602 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9472519e-1747-4489-b4a2-781161838773 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:56:58.764151 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9472519e-1747-4489-b4a2-781161838773 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:58.764423 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9472519e-1747-4489-b4a2-781161838773 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:58.764567 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e194dbf5-244a-4576-89d2-721259a7a749 None None] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:58.764731 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9472519e-1747-4489-b4a2-781161838773 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:58.764824 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e194dbf5-244a-4576-89d2-721259a7a749 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:58.765065 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e194dbf5-244a-4576-89d2-721259a7a749 None None] PATH_INFO: `/` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:58.766878 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 108/220] 10.222.0.62 () {54 vars in 927 bytes} [Wed Aug 24 05:56:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:56:58.772152 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e194dbf5-244a-4576-89d2-721259a7a749 None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:56:58.812066 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:56:58.842329 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9472519e-1747-4489-b4a2-781161838773 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:56:58.843798 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9472519e-1747-4489-b4a2-781161838773 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:56:58.919466 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e194dbf5-244a-4576-89d2-721259a7a749 None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:56:58.952025 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:56:58.957852 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4805f6-89df-4734-901e-e9783ca2ee13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:56:58.958564 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4805f6-89df-4734-901e-e9783ca2ee13 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:58.958906 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4805f6-89df-4734-901e-e9783ca2ee13 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:58.959240 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4805f6-89df-4734-901e-e9783ca2ee13 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:58.960589 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 113/221] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Aug 24 05:56:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 05:56:58.962262 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2c82f0-d635-4721-8c3f-d3731e99cd30 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:58.962873 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2c82f0-d635-4721-8c3f-d3731e99cd30 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:58.963148 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2c82f0-d635-4721-8c3f-d3731e99cd30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:58.971120 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4805f6-89df-4734-901e-e9783ca2ee13 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:56:58.977331 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application [None req-3d4805f6-89df-4734-901e-e9783ca2ee13 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 05:56:58.977331 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:56:58.977331 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:56:58.977331 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:56:58.977331 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:56:58.977331 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:56:58.977331 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:56:58.977331 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:56:58.977331 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:56:58.977331 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:56:58.977331 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:56:58.977331 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:56:58.977331 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:56:58.977331 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:56:58.977331 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:56:58.977331 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:56:58.978258 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:56:58.978258 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:56:58.978258 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:56:58.978258 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:56:58.978258 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:56:58.978258 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:56:58.978258 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:56:58.978258 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:56:58.978258 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:56:58.978258 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:56:58.978258 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:56:58.978258 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application with Lock( Aug 24 05:56:58.978258 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:56:58.978258 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:56:58.978258 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:56:58.978258 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:56:58.979031 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:56:58.979031 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:56:58.979031 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:56:58.979031 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:56:58.979031 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:56:58.979031 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:56:58.979031 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:56:58.979031 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:56:58.979031 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:56:58.979031 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:56:58.979031 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 05:56:58.979031 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Aug 24 05:56:58.979031 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 109/222] 10.222.0.62 () {56 vars in 1096 bytes} [Wed Aug 24 05:56:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:56:58.979706 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4805f6-89df-4734-901e-e9783ca2ee13 None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:56:58.982690 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-3c2c82f0-d635-4721-8c3f-d3731e99cd30 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:56:58.984463 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-3c2c82f0-d635-4721-8c3f-d3731e99cd30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:56:59.070184 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 114/223] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Aug 24 05:56:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 05:56:59.071914 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2c82f0-d635-4721-8c3f-d3731e99cd30 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:56:59.084568 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:56:59.109988 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4805f6-89df-4734-901e-e9783ca2ee13 None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:56:59.156327 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0146616-b14a-4f53-985e-177e724b3109 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:56:59.156997 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0146616-b14a-4f53-985e-177e724b3109 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:59.157278 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0146616-b14a-4f53-985e-177e724b3109 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:59.157583 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0146616-b14a-4f53-985e-177e724b3109 None admin] PATH_INFO: `/v3/projects/9391bbe626b44799942ef25c41bc33ad` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:59.159742 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0146616-b14a-4f53-985e-177e724b3109 None admin] RBAC: Authorizing `identity:get_project(project_id=9391bbe626b44799942ef25c41bc33ad)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:56:59.161691 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0146616-b14a-4f53-985e-177e724b3109 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:56:59.164733 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 110/224] 10.222.0.62 () {56 vars in 1186 bytes} [Wed Aug 24 05:56:58 2022] GET /identity/v3/projects/9391bbe626b44799942ef25c41bc33ad => generated 302 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:56:59.169921 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0146616-b14a-4f53-985e-177e724b3109 None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:56:59.187168 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2c82f0-d635-4721-8c3f-d3731e99cd30 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:56:59.215467 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:56:59.227759 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a5ad1e-50b0-48cf-bc0a-900a525fc00c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:56:59.228388 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a5ad1e-50b0-48cf-bc0a-900a525fc00c None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:59.230192 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a5ad1e-50b0-48cf-bc0a-900a525fc00c None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:59.230192 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a5ad1e-50b0-48cf-bc0a-900a525fc00c None admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:59.230491 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a5ad1e-50b0-48cf-bc0a-900a525fc00c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:56:59.230950 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a5ad1e-50b0-48cf-bc0a-900a525fc00c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:56:59.232670 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a5ad1e-50b0-48cf-bc0a-900a525fc00c None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:56:59.245343 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 115/225] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Aug 24 05:56:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:56:59.251043 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a5ad1e-50b0-48cf-bc0a-900a525fc00c None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:56:59.331486 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0146616-b14a-4f53-985e-177e724b3109 None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:56:59.396474 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc919792-83c1-4664-ace6-360a381d2367 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:56:59.397097 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc919792-83c1-4664-ace6-360a381d2367 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:59.397370 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc919792-83c1-4664-ace6-360a381d2367 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:59.397638 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc919792-83c1-4664-ace6-360a381d2367 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:59.398351 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc919792-83c1-4664-ace6-360a381d2367 None admin] RBAC: Adding query filter params (role.id=580aa7dbb38d4cff8f9d1bb8983f99ef, scope.project.id=9391bbe626b44799942ef25c41bc33ad, user.id=6e01f2a8b34743a9b2a7cc1d67be9890) {{(pid=81851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:56:59.398651 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc919792-83c1-4664-ace6-360a381d2367 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:56:59.400085 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc919792-83c1-4664-ace6-360a381d2367 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:56:59.412973 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 111/226] 10.222.0.62 () {56 vars in 1374 bytes} [Wed Aug 24 05:56:59 2022] GET /identity/v3/role_assignments?user.id=6e01f2a8b34743a9b2a7cc1d67be9890&scope.project.id=9391bbe626b44799942ef25c41bc33ad&role.id=580aa7dbb38d4cff8f9d1bb8983f99ef => generated 259 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:56:59.435602 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a5ad1e-50b0-48cf-bc0a-900a525fc00c None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:56:59.468875 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc919792-83c1-4664-ace6-360a381d2367 None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:56:59.508874 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:56:59.512731 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcfa384-ebd7-45ed-9ac6-75944cdb5e7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:56:59.513385 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcfa384-ebd7-45ed-9ac6-75944cdb5e7f None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:59.513664 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcfa384-ebd7-45ed-9ac6-75944cdb5e7f None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:59.513938 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcfa384-ebd7-45ed-9ac6-75944cdb5e7f None admin] PATH_INFO: `/v3/users/26afd7e3fe114d88a1d0e06e544c91eb` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:59.515084 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dcfa384-ebd7-45ed-9ac6-75944cdb5e7f None admin] RBAC: Authorizing `identity:get_user(user_id=26afd7e3fe114d88a1d0e06e544c91eb)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:56:59.516784 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dcfa384-ebd7-45ed-9ac6-75944cdb5e7f None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:56:59.521961 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 116/227] 10.222.0.62 () {56 vars in 1177 bytes} [Wed Aug 24 05:56:59 2022] GET /identity/v3/users/26afd7e3fe114d88a1d0e06e544c91eb => generated 283 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:56:59.524226 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcfa384-ebd7-45ed-9ac6-75944cdb5e7f None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:56:59.606785 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:56:59.643930 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc919792-83c1-4664-ace6-360a381d2367 None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:56:59.666677 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcfa384-ebd7-45ed-9ac6-75944cdb5e7f None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:56:59.706955 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddb98d7-468f-4f33-94e4-ebbe18e83ff2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:56:59.707916 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddb98d7-468f-4f33-94e4-ebbe18e83ff2 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:59.707916 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddb98d7-468f-4f33-94e4-ebbe18e83ff2 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:59.708146 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddb98d7-468f-4f33-94e4-ebbe18e83ff2 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:59.731341 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddb98d7-468f-4f33-94e4-ebbe18e83ff2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:56:59.736040 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application [None req-dddb98d7-468f-4f33-94e4-ebbe18e83ff2 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 05:56:59.736040 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:56:59.736040 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:56:59.736040 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:56:59.736040 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:56:59.736040 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:56:59.736040 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:56:59.736040 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:56:59.736040 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:56:59.736040 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:56:59.736040 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:56:59.736040 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:56:59.736040 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:56:59.736040 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:56:59.736040 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:56:59.736040 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:56:59.736981 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:56:59.736981 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:56:59.736981 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:56:59.736981 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:56:59.736981 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:56:59.736981 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:56:59.736981 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:56:59.736981 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:56:59.736981 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:56:59.736981 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:56:59.736981 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:56:59.736981 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application with Lock( Aug 24 05:56:59.736981 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:56:59.736981 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:56:59.736981 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:56:59.736981 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:56:59.737766 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:56:59.737766 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:56:59.737766 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:56:59.737766 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:56:59.737766 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:56:59.737766 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:56:59.737766 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:56:59.737766 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:56:59.737766 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:56:59.737766 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:56:59.737766 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 05:56:59.737766 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Aug 24 05:56:59.740060 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbb5e72-1705-49f8-bb8f-5ce83fcd1a47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:56:59.740712 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbb5e72-1705-49f8-bb8f-5ce83fcd1a47 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:59.740987 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbb5e72-1705-49f8-bb8f-5ce83fcd1a47 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:59.741308 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbb5e72-1705-49f8-bb8f-5ce83fcd1a47 None admin] PATH_INFO: `/v3/projects/9391bbe626b44799942ef25c41bc33ad` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:59.742907 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbb5e72-1705-49f8-bb8f-5ce83fcd1a47 None admin] RBAC: Authorizing `identity:get_project(project_id=9391bbe626b44799942ef25c41bc33ad)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:56:59.744340 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbb5e72-1705-49f8-bb8f-5ce83fcd1a47 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:56:59.746082 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 112/228] 10.222.0.62 () {56 vars in 1114 bytes} [Wed Aug 24 05:56:59 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 05:56:59.746631 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 117/229] 10.222.0.62 () {56 vars in 1186 bytes} [Wed Aug 24 05:56:59 2022] GET /identity/v3/projects/9391bbe626b44799942ef25c41bc33ad => generated 302 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:56:59.753537 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddb98d7-468f-4f33-94e4-ebbe18e83ff2 None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:56:59.754521 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbb5e72-1705-49f8-bb8f-5ce83fcd1a47 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:56:59.902081 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:56:59.951222 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbb5e72-1705-49f8-bb8f-5ce83fcd1a47 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:56:59.952579 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddb98d7-468f-4f33-94e4-ebbe18e83ff2 None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:56:59.976574 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:56:59.991683 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8557e5f-83da-4b51-813b-5f7ef0d76b88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:56:59.992318 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8557e5f-83da-4b51-813b-5f7ef0d76b88 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:59.992622 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8557e5f-83da-4b51-813b-5f7ef0d76b88 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:59.992896 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8557e5f-83da-4b51-813b-5f7ef0d76b88 None admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:59.993641 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8557e5f-83da-4b51-813b-5f7ef0d76b88 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:56:59.993945 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8557e5f-83da-4b51-813b-5f7ef0d76b88 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:56:59.995346 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8557e5f-83da-4b51-813b-5f7ef0d76b88 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:00.015198 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 118/230] 10.222.0.62 () {56 vars in 1111 bytes} [Wed Aug 24 05:56:59 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:57:00.016472 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8557e5f-83da-4b51-813b-5f7ef0d76b88 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:00.032729 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4eb0d2c-282c-4743-80a5-152da47b30da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:00.033432 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4eb0d2c-282c-4743-80a5-152da47b30da None admin] REQUEST_METHOD: `PUT` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:00.034708 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4eb0d2c-282c-4743-80a5-152da47b30da None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:00.034992 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4eb0d2c-282c-4743-80a5-152da47b30da None admin] PATH_INFO: `/v3/projects/9391bbe626b44799942ef25c41bc33ad/users/26afd7e3fe114d88a1d0e06e544c91eb/roles/49e47cd157d743f6905f1db07ec878a7` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:00.039118 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4eb0d2c-282c-4743-80a5-152da47b30da None admin] RBAC: Authorizing `identity:create_grant(project_id=9391bbe626b44799942ef25c41bc33ad, user_id=26afd7e3fe114d88a1d0e06e544c91eb, role_id=49e47cd157d743f6905f1db07ec878a7)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:00.040597 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4eb0d2c-282c-4743-80a5-152da47b30da None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:00.132764 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 113/231] 10.222.0.62 () {58 vars in 1439 bytes} [Wed Aug 24 05:56:59 2022] PUT /identity/v3/projects/9391bbe626b44799942ef25c41bc33ad/users/26afd7e3fe114d88a1d0e06e544c91eb/roles/49e47cd157d743f6905f1db07ec878a7 => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:57:00.154792 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8557e5f-83da-4b51-813b-5f7ef0d76b88 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:00.202785 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7677d07a-7a88-4258-9b32-17cd48eec17f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:00.203442 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7677d07a-7a88-4258-9b32-17cd48eec17f None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:00.203715 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7677d07a-7a88-4258-9b32-17cd48eec17f None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:00.203987 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7677d07a-7a88-4258-9b32-17cd48eec17f None admin] PATH_INFO: `/v3/users/26afd7e3fe114d88a1d0e06e544c91eb` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:00.205332 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7677d07a-7a88-4258-9b32-17cd48eec17f None admin] RBAC: Authorizing `identity:get_user(user_id=26afd7e3fe114d88a1d0e06e544c91eb)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:00.207281 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7677d07a-7a88-4258-9b32-17cd48eec17f None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:00.208919 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 119/232] 10.222.0.62 () {56 vars in 1177 bytes} [Wed Aug 24 05:57:00 2022] GET /identity/v3/users/26afd7e3fe114d88a1d0e06e544c91eb => generated 283 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:57:00.214541 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4eb0d2c-282c-4743-80a5-152da47b30da None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:00.358469 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4eb0d2c-282c-4743-80a5-152da47b30da None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:00.396815 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9092a243-2a06-43e5-9472-bd3bdfefd84f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:00.397796 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9092a243-2a06-43e5-9472-bd3bdfefd84f None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:00.398297 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9092a243-2a06-43e5-9472-bd3bdfefd84f None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:00.398764 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9092a243-2a06-43e5-9472-bd3bdfefd84f None admin] PATH_INFO: `/v3/projects/9391bbe626b44799942ef25c41bc33ad` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:00.400095 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9092a243-2a06-43e5-9472-bd3bdfefd84f None admin] RBAC: Authorizing `identity:get_project(project_id=9391bbe626b44799942ef25c41bc33ad)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:00.401747 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9092a243-2a06-43e5-9472-bd3bdfefd84f None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:00.403653 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 114/233] 10.222.0.62 () {56 vars in 1186 bytes} [Wed Aug 24 05:57:00 2022] GET /identity/v3/projects/9391bbe626b44799942ef25c41bc33ad => generated 302 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:57:00.409759 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7677d07a-7a88-4258-9b32-17cd48eec17f None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:00.495943 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:00.541356 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7677d07a-7a88-4258-9b32-17cd48eec17f None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:00.578773 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0f9247-8749-4456-9047-598dc14c7af0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:00.579442 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0f9247-8749-4456-9047-598dc14c7af0 None admin] REQUEST_METHOD: `PUT` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:00.579719 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0f9247-8749-4456-9047-598dc14c7af0 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:00.579992 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0f9247-8749-4456-9047-598dc14c7af0 None admin] PATH_INFO: `/v3/projects/9391bbe626b44799942ef25c41bc33ad/users/26afd7e3fe114d88a1d0e06e544c91eb/roles/ba3c2c3cb98c424f9ece19903a59c53c` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:00.582308 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0f9247-8749-4456-9047-598dc14c7af0 None admin] RBAC: Authorizing `identity:create_grant(project_id=9391bbe626b44799942ef25c41bc33ad, user_id=26afd7e3fe114d88a1d0e06e544c91eb, role_id=ba3c2c3cb98c424f9ece19903a59c53c)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:00.583773 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0f9247-8749-4456-9047-598dc14c7af0 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:00.719443 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a8a3fe-9e18-436f-a3ff-e78bb54be708 None None] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:00.719964 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a8a3fe-9e18-436f-a3ff-e78bb54be708 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:00.719964 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a8a3fe-9e18-436f-a3ff-e78bb54be708 None None] PATH_INFO: `/` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:00.722152 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 115/234] 10.222.0.62 () {54 vars in 927 bytes} [Wed Aug 24 05:57:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:57:00.728907 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ec6dc9-54a3-4520-8c89-594ef2ff459d None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:00.729190 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ec6dc9-54a3-4520-8c89-594ef2ff459d None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:00.729469 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ec6dc9-54a3-4520-8c89-594ef2ff459d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:00.729713 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 120/235] 10.222.0.62 () {58 vars in 1439 bytes} [Wed Aug 24 05:57:00 2022] PUT /identity/v3/projects/9391bbe626b44799942ef25c41bc33ad/users/26afd7e3fe114d88a1d0e06e544c91eb/roles/ba3c2c3cb98c424f9ece19903a59c53c => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:57:00.736207 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:00.750537 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-c5ec6dc9-54a3-4520-8c89-594ef2ff459d None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:57:00.751948 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-c5ec6dc9-54a3-4520-8c89-594ef2ff459d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:57:00.839306 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 116/236] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Aug 24 05:57:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 05:57:00.894106 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddfcac1-87af-423e-9bb3-67611067fa1e None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:00.894366 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddfcac1-87af-423e-9bb3-67611067fa1e None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:00.894708 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddfcac1-87af-423e-9bb3-67611067fa1e None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:00.896475 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 121/237] 10.222.0.62 () {54 vars in 927 bytes} [Wed Aug 24 05:57:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:57:00.901701 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1eeff5-2797-4ea4-8f96-3a6620bb589c None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:00.901994 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1eeff5-2797-4ea4-8f96-3a6620bb589c None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:00.902245 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1eeff5-2797-4ea4-8f96-3a6620bb589c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:00.929693 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-9e1eeff5-2797-4ea4-8f96-3a6620bb589c None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:57:00.930995 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-9e1eeff5-2797-4ea4-8f96-3a6620bb589c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:57:01.004127 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 117/238] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Aug 24 05:57:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 05:57:01.147271 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddfcac1-87af-423e-9bb3-67611067fa1e None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:01.272868 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddfcac1-87af-423e-9bb3-67611067fa1e None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:01.287652 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:01.298119 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1eeff5-2797-4ea4-8f96-3a6620bb589c None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:01.321789 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5ce5bd-6ad8-4f0f-982d-1c136bc93258 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:01.322434 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5ce5bd-6ad8-4f0f-982d-1c136bc93258 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:01.322712 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5ce5bd-6ad8-4f0f-982d-1c136bc93258 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:01.322986 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5ce5bd-6ad8-4f0f-982d-1c136bc93258 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:01.341214 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5ce5bd-6ad8-4f0f-982d-1c136bc93258 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:57:01.347000 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-9b5ce5bd-6ad8-4f0f-982d-1c136bc93258 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 05:57:01.347000 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:57:01.347000 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:57:01.347000 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:57:01.347000 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:57:01.347000 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:57:01.347000 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:57:01.347000 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:57:01.347000 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:57:01.347000 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:57:01.347000 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:57:01.347000 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:57:01.347000 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:57:01.347000 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:57:01.347000 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:57:01.347000 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:57:01.349659 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:57:01.349659 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:57:01.349659 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:57:01.349659 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:57:01.349659 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:57:01.349659 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:57:01.349659 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:57:01.349659 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:57:01.349659 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:57:01.349659 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:57:01.349659 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:57:01.349659 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 24 05:57:01.349659 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:57:01.349659 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:57:01.349659 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:57:01.349659 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:57:01.351553 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:57:01.351553 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:57:01.351553 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:57:01.351553 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:57:01.351553 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:57:01.351553 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:57:01.351553 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:57:01.351553 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:57:01.351553 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:57:01.351553 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:57:01.351553 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 05:57:01.351553 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 24 05:57:01.353067 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 122/239] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Aug 24 05:57:01 2022] GET /identity/v3/roles/reader => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:57:01.359477 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5ce5bd-6ad8-4f0f-982d-1c136bc93258 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:01.367165 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:01.414656 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1eeff5-2797-4ea4-8f96-3a6620bb589c None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:01.470568 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f8d84e-a236-4360-b351-820c2a22e232 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:01.471245 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f8d84e-a236-4360-b351-820c2a22e232 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:01.471525 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f8d84e-a236-4360-b351-820c2a22e232 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:01.471795 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f8d84e-a236-4360-b351-820c2a22e232 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:01.477249 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f8d84e-a236-4360-b351-820c2a22e232 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:01.478720 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f8d84e-a236-4360-b351-820c2a22e232 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:01.484598 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 118/240] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Aug 24 05:57:01 2022] GET /identity/v3/domains/default => generated 199 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:57:01.497306 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f8d84e-a236-4360-b351-820c2a22e232 None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:01.509149 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5ce5bd-6ad8-4f0f-982d-1c136bc93258 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:01.545175 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5d541d-a7e2-4257-a856-edd82189b73d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:01.546364 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5d541d-a7e2-4257-a856-edd82189b73d None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:01.546834 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5d541d-a7e2-4257-a856-edd82189b73d None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:01.547297 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5d541d-a7e2-4257-a856-edd82189b73d None admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:01.548142 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5d541d-a7e2-4257-a856-edd82189b73d None admin] RBAC: Adding query filter params (name=reader) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:57:01.548660 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5d541d-a7e2-4257-a856-edd82189b73d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:01.550246 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5d541d-a7e2-4257-a856-edd82189b73d None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:01.559260 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 123/241] 10.222.0.62 () {56 vars in 1101 bytes} [Wed Aug 24 05:57:01 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:57:01.565333 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5d541d-a7e2-4257-a856-edd82189b73d None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:01.601418 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:01.670033 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5d541d-a7e2-4257-a856-edd82189b73d None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:01.721288 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b009d0db-c073-4e1e-bab7-74bdbb818bbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:01.721934 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b009d0db-c073-4e1e-bab7-74bdbb818bbf None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:01.722209 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b009d0db-c073-4e1e-bab7-74bdbb818bbf None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:01.722528 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b009d0db-c073-4e1e-bab7-74bdbb818bbf None admin] PATH_INFO: `/v3/users/54248dd26f694bbe851c6f0963a9e349` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:01.745417 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b009d0db-c073-4e1e-bab7-74bdbb818bbf None admin] RBAC: Authorizing `identity:get_user(user_id=54248dd26f694bbe851c6f0963a9e349)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:01.746803 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b009d0db-c073-4e1e-bab7-74bdbb818bbf None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:01.749534 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 124/242] 10.222.0.62 () {56 vars in 1177 bytes} [Wed Aug 24 05:57:01 2022] GET /identity/v3/users/54248dd26f694bbe851c6f0963a9e349 => generated 297 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:57:01.756055 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b009d0db-c073-4e1e-bab7-74bdbb818bbf None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:01.759635 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f8d84e-a236-4360-b351-820c2a22e232 None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:01.811629 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:01.824620 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d03494-0fa0-46bf-87a4-fa52821ae5b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:01.825227 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d03494-0fa0-46bf-87a4-fa52821ae5b1 None admin] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:01.825501 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d03494-0fa0-46bf-87a4-fa52821ae5b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:01.825767 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d03494-0fa0-46bf-87a4-fa52821ae5b1 None admin] PATH_INFO: `/v3/users` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:01.826408 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d03494-0fa0-46bf-87a4-fa52821ae5b1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:01.827888 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d03494-0fa0-46bf-87a4-fa52821ae5b1 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:01.896804 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:01.903261 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b009d0db-c073-4e1e-bab7-74bdbb818bbf None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:01.910910 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 119/243] 10.222.0.62 () {60 vars in 1132 bytes} [Wed Aug 24 05:57:01 2022] POST /identity/v3/users => generated 293 bytes in 414 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 05:57:01.912543 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898c1f6c-efb9-433f-aa2b-78838af378c8 None None] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:01.912997 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898c1f6c-efb9-433f-aa2b-78838af378c8 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:01.913575 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898c1f6c-efb9-433f-aa2b-78838af378c8 None None] PATH_INFO: `/` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:01.915166 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 120/244] 10.222.0.62 () {54 vars in 927 bytes} [Wed Aug 24 05:57:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:57:01.922082 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c317975-97bd-40cd-9ddb-d3e0e7aa99ac None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:01.923301 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c317975-97bd-40cd-9ddb-d3e0e7aa99ac None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:01.925474 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c317975-97bd-40cd-9ddb-d3e0e7aa99ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:01.965194 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-4c317975-97bd-40cd-9ddb-d3e0e7aa99ac None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:57:01.967721 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-4c317975-97bd-40cd-9ddb-d3e0e7aa99ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:57:01.983928 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6b99dc-9763-4b14-9fa3-63d7f8cc30d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:01.985036 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6b99dc-9763-4b14-9fa3-63d7f8cc30d9 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:01.985523 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6b99dc-9763-4b14-9fa3-63d7f8cc30d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:01.986021 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6b99dc-9763-4b14-9fa3-63d7f8cc30d9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:02.073556 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6b99dc-9763-4b14-9fa3-63d7f8cc30d9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:02.075101 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6b99dc-9763-4b14-9fa3-63d7f8cc30d9 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:02.132918 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 121/245] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Aug 24 05:57:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 05:57:02.183530 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:02.191360 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 125/246] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Aug 24 05:57:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 05:57:02.197355 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c317975-97bd-40cd-9ddb-d3e0e7aa99ac None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:02.303747 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c317975-97bd-40cd-9ddb-d3e0e7aa99ac None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:02.320704 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:02.341535 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a3a2f4-7587-47d4-b6db-3bd54cc2c207 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:02.342094 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a3a2f4-7587-47d4-b6db-3bd54cc2c207 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:02.342373 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a3a2f4-7587-47d4-b6db-3bd54cc2c207 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:02.342711 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a3a2f4-7587-47d4-b6db-3bd54cc2c207 None admin] PATH_INFO: `/v3/projects/9391bbe626b44799942ef25c41bc33ad` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:02.343824 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a3a2f4-7587-47d4-b6db-3bd54cc2c207 None admin] RBAC: Authorizing `identity:get_project(project_id=9391bbe626b44799942ef25c41bc33ad)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:02.345368 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a3a2f4-7587-47d4-b6db-3bd54cc2c207 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:02.347460 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 122/247] 10.222.0.62 () {56 vars in 1186 bytes} [Wed Aug 24 05:57:02 2022] GET /identity/v3/projects/9391bbe626b44799942ef25c41bc33ad => generated 302 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:57:02.353290 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6b99dc-9763-4b14-9fa3-63d7f8cc30d9 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:02.458383 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a3a2f4-7587-47d4-b6db-3bd54cc2c207 None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:02.468350 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6b99dc-9763-4b14-9fa3-63d7f8cc30d9 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:02.514466 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d9ae73-9c99-44f6-85bf-b8cb46cc423f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:02.515413 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d9ae73-9c99-44f6-85bf-b8cb46cc423f None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:02.515970 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d9ae73-9c99-44f6-85bf-b8cb46cc423f None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:02.516248 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d9ae73-9c99-44f6-85bf-b8cb46cc423f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:02.517053 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d9ae73-9c99-44f6-85bf-b8cb46cc423f None admin] RBAC: Adding query filter params (role.id=e2f40474c6de4f5fadfa7efbe4087971, scope.project.id=9391bbe626b44799942ef25c41bc33ad, user.id=54248dd26f694bbe851c6f0963a9e349) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:57:02.517357 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d9ae73-9c99-44f6-85bf-b8cb46cc423f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:02.519973 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d9ae73-9c99-44f6-85bf-b8cb46cc423f None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:02.532932 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 126/248] 10.222.0.62 () {56 vars in 1374 bytes} [Wed Aug 24 05:57:02 2022] GET /identity/v3/role_assignments?user.id=54248dd26f694bbe851c6f0963a9e349&scope.project.id=9391bbe626b44799942ef25c41bc33ad&role.id=e2f40474c6de4f5fadfa7efbe4087971 => generated 259 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:57:02.534536 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a97e31f-ba90-45d0-825a-af711d640079 None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:02.534741 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a97e31f-ba90-45d0-825a-af711d640079 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:02.535019 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a97e31f-ba90-45d0-825a-af711d640079 None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:02.536642 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 127/249] 10.222.0.62 () {54 vars in 927 bytes} [Wed Aug 24 05:57:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:57:02.542480 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8adf74-51ff-4a3c-af8f-5224bf18cecf None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:02.542758 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8adf74-51ff-4a3c-af8f-5224bf18cecf None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:02.543027 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8adf74-51ff-4a3c-af8f-5224bf18cecf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:02.566687 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-ba8adf74-51ff-4a3c-af8f-5224bf18cecf None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:57:02.568373 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-ba8adf74-51ff-4a3c-af8f-5224bf18cecf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:57:02.593939 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a3a2f4-7587-47d4-b6db-3bd54cc2c207 None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:02.639381 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:02.650183 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef6b3d9-d458-4d39-854b-9eaf26572b9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:02.650891 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef6b3d9-d458-4d39-854b-9eaf26572b9d None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:02.651217 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef6b3d9-d458-4d39-854b-9eaf26572b9d None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:02.655068 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef6b3d9-d458-4d39-854b-9eaf26572b9d None admin] PATH_INFO: `/v3/roles/member` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:02.660254 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 128/250] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Aug 24 05:57:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 05:57:02.669146 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef6b3d9-d458-4d39-854b-9eaf26572b9d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:57:02.675979 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application [None req-aef6b3d9-d458-4d39-854b-9eaf26572b9d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 05:57:02.675979 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:57:02.675979 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:57:02.675979 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:57:02.675979 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:57:02.675979 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:57:02.675979 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:57:02.675979 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:57:02.675979 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:57:02.675979 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:57:02.675979 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:57:02.675979 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:57:02.675979 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:57:02.675979 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:57:02.675979 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:57:02.675979 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:57:02.676951 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:57:02.676951 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:57:02.676951 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:57:02.676951 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:57:02.676951 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:57:02.676951 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:57:02.676951 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:57:02.676951 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:57:02.676951 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:57:02.676951 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:57:02.676951 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:57:02.676951 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application with Lock( Aug 24 05:57:02.676951 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:57:02.676951 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:57:02.676951 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:57:02.676951 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:57:02.677722 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:57:02.677722 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:57:02.677722 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:57:02.677722 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:57:02.677722 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:57:02.677722 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:57:02.677722 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:57:02.677722 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:57:02.677722 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:57:02.677722 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:57:02.677722 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 05:57:02.677722 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Aug 24 05:57:02.679741 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 123/251] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Aug 24 05:57:02 2022] GET /identity/v3/roles/member => generated 84 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 05:57:02.685775 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8adf74-51ff-4a3c-af8f-5224bf18cecf None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:02.797123 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8adf74-51ff-4a3c-af8f-5224bf18cecf None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:02.844110 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539bb9a6-b132-4943-8389-d2d9591df5c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:02.844670 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539bb9a6-b132-4943-8389-d2d9591df5c2 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:02.844946 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539bb9a6-b132-4943-8389-d2d9591df5c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:02.845266 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539bb9a6-b132-4943-8389-d2d9591df5c2 None admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:02.846716 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539bb9a6-b132-4943-8389-d2d9591df5c2 None admin] RBAC: Adding query filter params (name=member) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:57:02.847078 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539bb9a6-b132-4943-8389-d2d9591df5c2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:02.848458 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539bb9a6-b132-4943-8389-d2d9591df5c2 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:02.859886 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 129/252] 10.222.0.62 () {56 vars in 1101 bytes} [Wed Aug 24 05:57:02 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:57:02.870597 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef6b3d9-d458-4d39-854b-9eaf26572b9d None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:02.941278 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:02.949466 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539bb9a6-b132-4943-8389-d2d9591df5c2 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:03.032312 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef6b3d9-d458-4d39-854b-9eaf26572b9d None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:03.044740 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:03.097774 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297f5d85-438f-4860-9fb7-a69e3a67dec3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:03.098754 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297f5d85-438f-4860-9fb7-a69e3a67dec3 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:03.099250 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297f5d85-438f-4860-9fb7-a69e3a67dec3 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:03.099719 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297f5d85-438f-4860-9fb7-a69e3a67dec3 None admin] PATH_INFO: `/v3/users/6e01f2a8b34743a9b2a7cc1d67be9890` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:03.105226 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297f5d85-438f-4860-9fb7-a69e3a67dec3 None admin] RBAC: Authorizing `identity:get_user(user_id=6e01f2a8b34743a9b2a7cc1d67be9890)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:03.107214 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297f5d85-438f-4860-9fb7-a69e3a67dec3 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:03.109156 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 124/253] 10.222.0.62 () {56 vars in 1177 bytes} [Wed Aug 24 05:57:02 2022] GET /identity/v3/users/6e01f2a8b34743a9b2a7cc1d67be9890 => generated 297 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:57:03.114981 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297f5d85-438f-4860-9fb7-a69e3a67dec3 None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:03.157387 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539bb9a6-b132-4943-8389-d2d9591df5c2 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:03.224073 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cc712c-baa3-4aed-b2ac-1dce3c642880 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:03.227407 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cc712c-baa3-4aed-b2ac-1dce3c642880 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:03.227904 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cc712c-baa3-4aed-b2ac-1dce3c642880 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:03.228376 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cc712c-baa3-4aed-b2ac-1dce3c642880 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:03.240361 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57cc712c-baa3-4aed-b2ac-1dce3c642880 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:57:03.245974 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-57cc712c-baa3-4aed-b2ac-1dce3c642880 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 05:57:03.245974 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:57:03.245974 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:57:03.245974 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:57:03.245974 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:57:03.245974 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:57:03.245974 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:57:03.245974 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:57:03.245974 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:57:03.245974 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:57:03.245974 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:57:03.245974 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:57:03.245974 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:57:03.245974 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:57:03.245974 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:57:03.245974 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:57:03.246851 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:57:03.246851 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:57:03.246851 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:57:03.246851 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:57:03.246851 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:57:03.246851 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:57:03.246851 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:57:03.246851 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:57:03.246851 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:57:03.246851 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:57:03.246851 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:57:03.246851 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 24 05:57:03.246851 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:57:03.246851 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:57:03.246851 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:57:03.246851 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:57:03.247660 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:57:03.247660 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:57:03.247660 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:57:03.247660 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:57:03.247660 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:57:03.247660 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:57:03.247660 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:57:03.247660 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:57:03.247660 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:57:03.247660 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:57:03.247660 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 05:57:03.247660 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 24 05:57:03.250318 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 130/254] 10.222.0.62 () {56 vars in 1096 bytes} [Wed Aug 24 05:57:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:57:03.261439 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cc712c-baa3-4aed-b2ac-1dce3c642880 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:03.304985 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297f5d85-438f-4860-9fb7-a69e3a67dec3 None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:03.334548 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:03.375863 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b3b2a8-837e-493c-b4bc-7b69b52c8b6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:03.377794 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b3b2a8-837e-493c-b4bc-7b69b52c8b6f None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:03.378616 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b3b2a8-837e-493c-b4bc-7b69b52c8b6f None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:03.379168 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b3b2a8-837e-493c-b4bc-7b69b52c8b6f None admin] PATH_INFO: `/v3/projects/9391bbe626b44799942ef25c41bc33ad` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:03.385863 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b3b2a8-837e-493c-b4bc-7b69b52c8b6f None admin] RBAC: Authorizing `identity:get_project(project_id=9391bbe626b44799942ef25c41bc33ad)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:03.387861 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b3b2a8-837e-493c-b4bc-7b69b52c8b6f None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:03.393137 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 125/255] 10.222.0.62 () {56 vars in 1186 bytes} [Wed Aug 24 05:57:03 2022] GET /identity/v3/projects/9391bbe626b44799942ef25c41bc33ad => generated 302 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:57:03.395113 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9b5dad-65db-4bca-9c97-3e535b96ed6d None None] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:03.397030 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9b5dad-65db-4bca-9c97-3e535b96ed6d None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:03.397814 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9b5dad-65db-4bca-9c97-3e535b96ed6d None None] PATH_INFO: `/` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:03.406090 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 126/256] 10.222.0.62 () {54 vars in 927 bytes} [Wed Aug 24 05:57:03 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:57:03.407162 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9b5dad-65db-4bca-9c97-3e535b96ed6d None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:03.442859 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cc712c-baa3-4aed-b2ac-1dce3c642880 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:03.462841 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:03.512532 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b4bba9-f664-48c3-ae93-50e3849cb32a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:03.513152 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b4bba9-f664-48c3-ae93-50e3849cb32a None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:03.513428 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b4bba9-f664-48c3-ae93-50e3849cb32a None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:03.513697 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b4bba9-f664-48c3-ae93-50e3849cb32a None admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:03.514352 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b4bba9-f664-48c3-ae93-50e3849cb32a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:57:03.514651 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b4bba9-f664-48c3-ae93-50e3849cb32a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:03.516773 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b4bba9-f664-48c3-ae93-50e3849cb32a None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:03.539988 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 131/257] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Aug 24 05:57:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:57:03.541255 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae16f1d6-291f-4acf-8730-6aa6dd23c88f None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:03.541532 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae16f1d6-291f-4acf-8730-6aa6dd23c88f None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:03.541801 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae16f1d6-291f-4acf-8730-6aa6dd23c88f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:03.573674 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-ae16f1d6-291f-4acf-8730-6aa6dd23c88f None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:57:03.574859 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-ae16f1d6-291f-4acf-8730-6aa6dd23c88f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:57:03.594853 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9b5dad-65db-4bca-9c97-3e535b96ed6d None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:03.665634 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 132/258] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Aug 24 05:57:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 05:57:03.666328 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae16f1d6-291f-4acf-8730-6aa6dd23c88f None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:03.717209 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35339cbb-d400-45f4-9c74-f7278a1193e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:03.717898 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35339cbb-d400-45f4-9c74-f7278a1193e4 None admin] REQUEST_METHOD: `PUT` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:03.718177 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35339cbb-d400-45f4-9c74-f7278a1193e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:03.718451 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35339cbb-d400-45f4-9c74-f7278a1193e4 None admin] PATH_INFO: `/v3/projects/9391bbe626b44799942ef25c41bc33ad/users/6e01f2a8b34743a9b2a7cc1d67be9890/roles/580aa7dbb38d4cff8f9d1bb8983f99ef` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:03.729285 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35339cbb-d400-45f4-9c74-f7278a1193e4 None admin] RBAC: Authorizing `identity:create_grant(project_id=9391bbe626b44799942ef25c41bc33ad, user_id=6e01f2a8b34743a9b2a7cc1d67be9890, role_id=580aa7dbb38d4cff8f9d1bb8983f99ef)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:03.730744 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35339cbb-d400-45f4-9c74-f7278a1193e4 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:03.770034 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae16f1d6-291f-4acf-8730-6aa6dd23c88f None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:03.773412 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:03.818376 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daed7019-0e90-4820-a47a-698c8637c0a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:03.819028 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daed7019-0e90-4820-a47a-698c8637c0a1 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:03.819304 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daed7019-0e90-4820-a47a-698c8637c0a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:03.819573 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daed7019-0e90-4820-a47a-698c8637c0a1 None admin] PATH_INFO: `/v3/users/26afd7e3fe114d88a1d0e06e544c91eb` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:03.820728 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daed7019-0e90-4820-a47a-698c8637c0a1 None admin] RBAC: Authorizing `identity:get_user(user_id=26afd7e3fe114d88a1d0e06e544c91eb)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:03.822229 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daed7019-0e90-4820-a47a-698c8637c0a1 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:03.824121 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 133/259] 10.222.0.62 () {56 vars in 1177 bytes} [Wed Aug 24 05:57:03 2022] GET /identity/v3/users/26afd7e3fe114d88a1d0e06e544c91eb => generated 283 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:57:03.831555 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daed7019-0e90-4820-a47a-698c8637c0a1 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:03.846852 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 127/260] 10.222.0.62 () {58 vars in 1439 bytes} [Wed Aug 24 05:57:03 2022] PUT /identity/v3/projects/9391bbe626b44799942ef25c41bc33ad/users/6e01f2a8b34743a9b2a7cc1d67be9890/roles/580aa7dbb38d4cff8f9d1bb8983f99ef => generated 0 bytes in 440 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:57:03.963572 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daed7019-0e90-4820-a47a-698c8637c0a1 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:04.015576 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d12cd02-0dfd-422a-a642-65f5998efa34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:04.016646 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d12cd02-0dfd-422a-a642-65f5998efa34 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:04.017167 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d12cd02-0dfd-422a-a642-65f5998efa34 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:04.017696 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d12cd02-0dfd-422a-a642-65f5998efa34 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:04.072077 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:04.111178 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d12cd02-0dfd-422a-a642-65f5998efa34 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:04.113079 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d12cd02-0dfd-422a-a642-65f5998efa34 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:04.204724 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35339cbb-d400-45f4-9c74-f7278a1193e4 None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:04.242519 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 134/261] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Aug 24 05:57:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 24 05:57:04.247828 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d12cd02-0dfd-422a-a642-65f5998efa34 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:04.282219 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:04.371153 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:04.374728 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35339cbb-d400-45f4-9c74-f7278a1193e4 None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:04.422748 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0477be-29f1-4d42-b40d-953cd3ad3da7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:04.424658 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0477be-29f1-4d42-b40d-953cd3ad3da7 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:04.425706 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0477be-29f1-4d42-b40d-953cd3ad3da7 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:04.426490 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0477be-29f1-4d42-b40d-953cd3ad3da7 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:04.436248 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d12cd02-0dfd-422a-a642-65f5998efa34 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:04.439524 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0477be-29f1-4d42-b40d-953cd3ad3da7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:57:04.446082 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application [None req-2a0477be-29f1-4d42-b40d-953cd3ad3da7 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 05:57:04.446082 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:57:04.446082 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:57:04.446082 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:57:04.446082 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:57:04.446082 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:57:04.446082 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:57:04.446082 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:57:04.446082 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:57:04.446082 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:57:04.446082 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:57:04.446082 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:57:04.446082 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:57:04.446082 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:57:04.446082 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:57:04.446082 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:57:04.447032 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:57:04.447032 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:57:04.447032 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:57:04.447032 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:57:04.447032 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:57:04.447032 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:57:04.447032 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:57:04.447032 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:57:04.447032 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:57:04.447032 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:57:04.447032 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:57:04.447032 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application with Lock( Aug 24 05:57:04.447032 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:57:04.447032 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:57:04.447032 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:57:04.447032 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:57:04.447816 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:57:04.447816 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:57:04.447816 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:57:04.447816 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:57:04.447816 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:57:04.447816 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:57:04.447816 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:57:04.447816 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:57:04.447816 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:57:04.447816 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:57:04.447816 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 05:57:04.447816 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Aug 24 05:57:04.450898 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 128/262] 10.222.0.62 () {56 vars in 1114 bytes} [Wed Aug 24 05:57:04 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 05:57:04.457133 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0477be-29f1-4d42-b40d-953cd3ad3da7 None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:04.526219 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193dae90-ec7d-40f0-9d1f-233a0ef3ec96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:04.527424 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193dae90-ec7d-40f0-9d1f-233a0ef3ec96 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:04.527840 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193dae90-ec7d-40f0-9d1f-233a0ef3ec96 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:04.528245 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193dae90-ec7d-40f0-9d1f-233a0ef3ec96 None admin] PATH_INFO: `/v3/projects/9391bbe626b44799942ef25c41bc33ad` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:04.533742 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193dae90-ec7d-40f0-9d1f-233a0ef3ec96 None admin] RBAC: Authorizing `identity:get_project(project_id=9391bbe626b44799942ef25c41bc33ad)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:04.535399 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193dae90-ec7d-40f0-9d1f-233a0ef3ec96 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:04.537255 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 135/263] 10.222.0.62 () {56 vars in 1186 bytes} [Wed Aug 24 05:57:04 2022] GET /identity/v3/projects/9391bbe626b44799942ef25c41bc33ad => generated 302 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:57:04.543697 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193dae90-ec7d-40f0-9d1f-233a0ef3ec96 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:04.619590 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:04.623720 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0477be-29f1-4d42-b40d-953cd3ad3da7 None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:04.699255 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11c57f6-7316-4c7d-817b-c7642bae95c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:04.700239 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11c57f6-7316-4c7d-817b-c7642bae95c0 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:04.700880 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11c57f6-7316-4c7d-817b-c7642bae95c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:04.701355 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11c57f6-7316-4c7d-817b-c7642bae95c0 None admin] PATH_INFO: `/v3/roles` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:04.702259 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11c57f6-7316-4c7d-817b-c7642bae95c0 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:57:04.702771 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11c57f6-7316-4c7d-817b-c7642bae95c0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:04.704634 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11c57f6-7316-4c7d-817b-c7642bae95c0 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:04.724517 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 129/264] 10.222.0.62 () {56 vars in 1111 bytes} [Wed Aug 24 05:57:04 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:57:04.730569 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11c57f6-7316-4c7d-817b-c7642bae95c0 None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:04.791705 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193dae90-ec7d-40f0-9d1f-233a0ef3ec96 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:04.875854 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87432040-19ab-4044-94c6-0cb4e30a0933 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:04.877510 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87432040-19ab-4044-94c6-0cb4e30a0933 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:04.877923 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87432040-19ab-4044-94c6-0cb4e30a0933 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:04.878315 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87432040-19ab-4044-94c6-0cb4e30a0933 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:04.879252 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87432040-19ab-4044-94c6-0cb4e30a0933 None admin] RBAC: Adding query filter params (role.id=49e47cd157d743f6905f1db07ec878a7, scope.project.id=9391bbe626b44799942ef25c41bc33ad, user.id=26afd7e3fe114d88a1d0e06e544c91eb) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:57:04.879717 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87432040-19ab-4044-94c6-0cb4e30a0933 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:04.881313 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87432040-19ab-4044-94c6-0cb4e30a0933 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:04.889276 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11c57f6-7316-4c7d-817b-c7642bae95c0 None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:04.893391 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 136/265] 10.222.0.62 () {56 vars in 1374 bytes} [Wed Aug 24 05:57:04 2022] GET /identity/v3/role_assignments?user.id=26afd7e3fe114d88a1d0e06e544c91eb&scope.project.id=9391bbe626b44799942ef25c41bc33ad&role.id=49e47cd157d743f6905f1db07ec878a7 => generated 608 bytes in 350 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:57:04.949298 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3b2e0a-1898-4a76-9c13-8ecc66b6b8ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:04.949947 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3b2e0a-1898-4a76-9c13-8ecc66b6b8ee None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:04.950285 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3b2e0a-1898-4a76-9c13-8ecc66b6b8ee None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:04.950566 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3b2e0a-1898-4a76-9c13-8ecc66b6b8ee None admin] PATH_INFO: `/v3/users/26afd7e3fe114d88a1d0e06e544c91eb` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:04.957468 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3b2e0a-1898-4a76-9c13-8ecc66b6b8ee None admin] RBAC: Authorizing `identity:get_user(user_id=26afd7e3fe114d88a1d0e06e544c91eb)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:04.958991 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3b2e0a-1898-4a76-9c13-8ecc66b6b8ee None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:04.960737 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 130/266] 10.222.0.62 () {56 vars in 1177 bytes} [Wed Aug 24 05:57:04 2022] GET /identity/v3/users/26afd7e3fe114d88a1d0e06e544c91eb => generated 283 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:57:04.966676 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87432040-19ab-4044-94c6-0cb4e30a0933 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:04.975206 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:05.121805 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87432040-19ab-4044-94c6-0cb4e30a0933 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:05.122604 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563cc1e0-2c94-41d8-aaa9-c435206c1a8a None None] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:05.122905 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563cc1e0-2c94-41d8-aaa9-c435206c1a8a None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:05.123462 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563cc1e0-2c94-41d8-aaa9-c435206c1a8a None None] PATH_INFO: `/` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:05.126183 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 131/267] 10.222.0.62 () {54 vars in 927 bytes} [Wed Aug 24 05:57:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:57:05.132275 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc4c27a-baf1-44ba-af41-0e4e3f11e215 None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:05.132593 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc4c27a-baf1-44ba-af41-0e4e3f11e215 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:05.132872 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc4c27a-baf1-44ba-af41-0e4e3f11e215 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:05.175345 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-ebc4c27a-baf1-44ba-af41-0e4e3f11e215 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:57:05.176557 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-ebc4c27a-baf1-44ba-af41-0e4e3f11e215 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:57:05.194994 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3910f83c-30a6-4579-8d3b-b3ccc4a4989e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:05.195616 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3910f83c-30a6-4579-8d3b-b3ccc4a4989e None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:05.195892 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3910f83c-30a6-4579-8d3b-b3ccc4a4989e None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:05.196163 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3910f83c-30a6-4579-8d3b-b3ccc4a4989e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:05.238922 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:05.293595 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3910f83c-30a6-4579-8d3b-b3ccc4a4989e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:05.295059 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3910f83c-30a6-4579-8d3b-b3ccc4a4989e None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:05.299921 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:05.305240 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 132/268] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Aug 24 05:57:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 05:57:05.306532 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9061d432-5a09-4207-a5db-989a7f616d5f None None] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:05.306781 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9061d432-5a09-4207-a5db-989a7f616d5f None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:05.307076 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9061d432-5a09-4207-a5db-989a7f616d5f None None] PATH_INFO: `/` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:05.308710 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 133/269] 10.222.0.62 () {54 vars in 927 bytes} [Wed Aug 24 05:57:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:57:05.314461 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4a68b8-da68-4614-adaa-6417ee39e80f None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:05.314745 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4a68b8-da68-4614-adaa-6417ee39e80f None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:05.315401 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4a68b8-da68-4614-adaa-6417ee39e80f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:05.341204 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-9e4a68b8-da68-4614-adaa-6417ee39e80f None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:57:05.343362 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-9e4a68b8-da68-4614-adaa-6417ee39e80f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:57:05.424886 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 137/270] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Aug 24 05:57:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 459 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 05:57:05.425689 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 134/271] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Aug 24 05:57:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 05:57:05.426315 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478315c7-dbcc-4770-bbf9-7054058eaac6 None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:05.426602 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478315c7-dbcc-4770-bbf9-7054058eaac6 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:05.431454 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478315c7-dbcc-4770-bbf9-7054058eaac6 None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:05.432178 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4a68b8-da68-4614-adaa-6417ee39e80f None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:05.432911 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 138/272] 10.222.0.62 () {54 vars in 927 bytes} [Wed Aug 24 05:57:05 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:57:05.438191 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d2197d-1f8d-4ee4-a720-3a7b920c1b02 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:05.438473 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d2197d-1f8d-4ee4-a720-3a7b920c1b02 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:05.438748 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d2197d-1f8d-4ee4-a720-3a7b920c1b02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:05.464145 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-51d2197d-1f8d-4ee4-a720-3a7b920c1b02 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:57:05.465600 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-51d2197d-1f8d-4ee4-a720-3a7b920c1b02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:57:05.577029 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 139/273] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Aug 24 05:57:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 05:57:05.594651 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:05.596378 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d2197d-1f8d-4ee4-a720-3a7b920c1b02 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:05.609037 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4a68b8-da68-4614-adaa-6417ee39e80f None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:05.645857 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:05.675718 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a577d4-885f-4210-a32f-33c66d44aad9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:05.676269 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a577d4-885f-4210-a32f-33c66d44aad9 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:05.676559 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a577d4-885f-4210-a32f-33c66d44aad9 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:05.676831 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a577d4-885f-4210-a32f-33c66d44aad9 None admin] PATH_INFO: `/v3/projects/9391bbe626b44799942ef25c41bc33ad` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:05.680525 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a577d4-885f-4210-a32f-33c66d44aad9 None admin] RBAC: Authorizing `identity:get_project(project_id=9391bbe626b44799942ef25c41bc33ad)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:05.681996 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a577d4-885f-4210-a32f-33c66d44aad9 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:05.685178 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 135/274] 10.222.0.62 () {56 vars in 1186 bytes} [Wed Aug 24 05:57:05 2022] GET /identity/v3/projects/9391bbe626b44799942ef25c41bc33ad => generated 302 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:57:05.689928 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a577d4-885f-4210-a32f-33c66d44aad9 None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:05.769983 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d2197d-1f8d-4ee4-a720-3a7b920c1b02 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:05.816986 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba725212-a0c0-4a86-a59b-4329556deae8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:05.817655 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba725212-a0c0-4a86-a59b-4329556deae8 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:05.817939 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba725212-a0c0-4a86-a59b-4329556deae8 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:05.818204 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba725212-a0c0-4a86-a59b-4329556deae8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:05.831752 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba725212-a0c0-4a86-a59b-4329556deae8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:57:05.837572 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a577d4-885f-4210-a32f-33c66d44aad9 None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:05.837845 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-ba725212-a0c0-4a86-a59b-4329556deae8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 05:57:05.837845 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:57:05.837845 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:57:05.837845 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:57:05.837845 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:57:05.837845 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:57:05.837845 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:57:05.837845 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:57:05.837845 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:57:05.837845 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:57:05.837845 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:57:05.837845 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:57:05.837845 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:57:05.837845 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:57:05.837845 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:57:05.837845 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:57:05.838707 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:57:05.838707 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:57:05.838707 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:57:05.838707 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:57:05.838707 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:57:05.838707 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:57:05.838707 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:57:05.838707 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:57:05.838707 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:57:05.838707 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:57:05.838707 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:57:05.838707 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 24 05:57:05.838707 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:57:05.838707 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:57:05.838707 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:57:05.838707 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:57:05.840080 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:57:05.840080 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:57:05.840080 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:57:05.840080 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:57:05.840080 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:57:05.840080 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:57:05.840080 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:57:05.840080 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:57:05.840080 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:57:05.840080 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:57:05.840080 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 05:57:05.840080 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 24 05:57:05.840080 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 140/275] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Aug 24 05:57:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 05:57:05.840791 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba725212-a0c0-4a86-a59b-4329556deae8 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:05.883333 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44705112-8677-4521-8abc-bc05066d7765 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:05.883964 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44705112-8677-4521-8abc-bc05066d7765 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:05.884226 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44705112-8677-4521-8abc-bc05066d7765 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:05.884516 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44705112-8677-4521-8abc-bc05066d7765 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:05.885646 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44705112-8677-4521-8abc-bc05066d7765 None admin] RBAC: Adding query filter params (role.id=ba3c2c3cb98c424f9ece19903a59c53c, scope.project.id=9391bbe626b44799942ef25c41bc33ad, user.id=26afd7e3fe114d88a1d0e06e544c91eb) {{(pid=81851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:57:05.885646 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44705112-8677-4521-8abc-bc05066d7765 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:05.887018 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44705112-8677-4521-8abc-bc05066d7765 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:05.893300 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 136/276] 10.222.0.62 () {56 vars in 1374 bytes} [Wed Aug 24 05:57:05 2022] GET /identity/v3/role_assignments?user.id=26afd7e3fe114d88a1d0e06e544c91eb&scope.project.id=9391bbe626b44799942ef25c41bc33ad&role.id=ba3c2c3cb98c424f9ece19903a59c53c => generated 608 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:57:05.894617 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44705112-8677-4521-8abc-bc05066d7765 None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:05.934619 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:05.957357 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:05.967535 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba725212-a0c0-4a86-a59b-4329556deae8 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:06.006016 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2158cf-e3dd-4840-9d41-b9f937289409 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:06.007044 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2158cf-e3dd-4840-9d41-b9f937289409 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:06.007503 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2158cf-e3dd-4840-9d41-b9f937289409 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:06.007952 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2158cf-e3dd-4840-9d41-b9f937289409 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:06.009155 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2158cf-e3dd-4840-9d41-b9f937289409 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:06.010632 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2158cf-e3dd-4840-9d41-b9f937289409 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:06.012246 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 141/277] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Aug 24 05:57:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:57:06.013079 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2158cf-e3dd-4840-9d41-b9f937289409 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:06.042002 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44705112-8677-4521-8abc-bc05066d7765 None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:06.083284 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c5fe02-2a6c-456c-b521-eecf5618ff3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:06.084212 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c5fe02-2a6c-456c-b521-eecf5618ff3e None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:06.084769 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c5fe02-2a6c-456c-b521-eecf5618ff3e None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:06.085044 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c5fe02-2a6c-456c-b521-eecf5618ff3e None admin] PATH_INFO: `/v3/roles` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:06.085917 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c5fe02-2a6c-456c-b521-eecf5618ff3e None admin] RBAC: Adding query filter params (name=member) {{(pid=81851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:57:06.086175 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c5fe02-2a6c-456c-b521-eecf5618ff3e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:06.087994 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c5fe02-2a6c-456c-b521-eecf5618ff3e None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:06.097219 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 137/278] 10.222.0.62 () {56 vars in 1101 bytes} [Wed Aug 24 05:57:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:57:06.097960 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c5fe02-2a6c-456c-b521-eecf5618ff3e None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:06.133415 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2158cf-e3dd-4840-9d41-b9f937289409 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:06.189677 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a127097f-d51a-4ed2-a3b2-07d2af3fc9a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:06.190570 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a127097f-d51a-4ed2-a3b2-07d2af3fc9a3 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:06.190855 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a127097f-d51a-4ed2-a3b2-07d2af3fc9a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:06.191121 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a127097f-d51a-4ed2-a3b2-07d2af3fc9a3 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:06.201145 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:06.202851 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a127097f-d51a-4ed2-a3b2-07d2af3fc9a3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:57:06.207441 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-a127097f-d51a-4ed2-a3b2-07d2af3fc9a3 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 05:57:06.207441 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:57:06.207441 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:57:06.207441 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:57:06.207441 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:57:06.207441 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:57:06.207441 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:57:06.207441 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:57:06.207441 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:57:06.207441 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:57:06.207441 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:57:06.207441 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:57:06.207441 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:57:06.207441 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:57:06.207441 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:57:06.207441 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:57:06.211369 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:57:06.211369 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:57:06.211369 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:57:06.211369 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:57:06.211369 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:57:06.211369 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:57:06.211369 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:57:06.211369 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:57:06.211369 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:57:06.211369 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:57:06.211369 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:57:06.211369 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 24 05:57:06.211369 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:57:06.211369 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:57:06.211369 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:57:06.211369 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:57:06.212162 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:57:06.212162 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:57:06.212162 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:57:06.212162 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:57:06.212162 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:57:06.212162 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:57:06.212162 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:57:06.212162 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:57:06.212162 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:57:06.212162 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:57:06.212162 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 05:57:06.212162 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 24 05:57:06.212162 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 142/279] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Aug 24 05:57:06 2022] GET /identity/v3/roles/reader => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:57:06.212162 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a127097f-d51a-4ed2-a3b2-07d2af3fc9a3 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:06.220018 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c5fe02-2a6c-456c-b521-eecf5618ff3e None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:06.235141 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:06.257272 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95864d87-5b96-4ec4-96e3-1839b02bbada None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:06.257882 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95864d87-5b96-4ec4-96e3-1839b02bbada None admin] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:06.258157 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95864d87-5b96-4ec4-96e3-1839b02bbada None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:06.258428 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95864d87-5b96-4ec4-96e3-1839b02bbada None admin] PATH_INFO: `/v3/users` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:06.259065 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95864d87-5b96-4ec4-96e3-1839b02bbada None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:06.260761 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95864d87-5b96-4ec4-96e3-1839b02bbada None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:06.314873 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a127097f-d51a-4ed2-a3b2-07d2af3fc9a3 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:06.349365 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6636a42-65ec-420f-848a-95cb77901896 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:06.349995 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6636a42-65ec-420f-848a-95cb77901896 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:06.350276 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6636a42-65ec-420f-848a-95cb77901896 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:06.350548 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6636a42-65ec-420f-848a-95cb77901896 None admin] PATH_INFO: `/v3/users/3e1d2954e5cd4a9c94939e2e0979bb08` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:06.365125 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6636a42-65ec-420f-848a-95cb77901896 None admin] RBAC: Authorizing `identity:get_user(user_id=3e1d2954e5cd4a9c94939e2e0979bb08)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:06.366719 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 138/280] 10.222.0.62 () {60 vars in 1132 bytes} [Wed Aug 24 05:57:06 2022] POST /identity/v3/users => generated 293 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:57:06.366847 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6636a42-65ec-420f-848a-95cb77901896 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:06.367603 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95864d87-5b96-4ec4-96e3-1839b02bbada None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:06.368658 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 143/281] 10.222.0.62 () {56 vars in 1177 bytes} [Wed Aug 24 05:57:06 2022] GET /identity/v3/users/3e1d2954e5cd4a9c94939e2e0979bb08 => generated 293 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:57:06.384525 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6636a42-65ec-420f-848a-95cb77901896 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:06.471651 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:06.503858 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6636a42-65ec-420f-848a-95cb77901896 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:06.504807 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95864d87-5b96-4ec4-96e3-1839b02bbada None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:06.546862 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984548bc-b0d8-44df-a1e3-b70433ccb295 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:06.547477 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984548bc-b0d8-44df-a1e3-b70433ccb295 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:06.547784 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984548bc-b0d8-44df-a1e3-b70433ccb295 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:06.547972 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4490d2-b411-4e37-85af-03ba6f2e3212 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:06.548080 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984548bc-b0d8-44df-a1e3-b70433ccb295 None admin] PATH_INFO: `/v3/roles` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:06.548617 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4490d2-b411-4e37-85af-03ba6f2e3212 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:06.549284 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984548bc-b0d8-44df-a1e3-b70433ccb295 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:57:06.549542 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984548bc-b0d8-44df-a1e3-b70433ccb295 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:06.549749 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4490d2-b411-4e37-85af-03ba6f2e3212 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:06.550021 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4490d2-b411-4e37-85af-03ba6f2e3212 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:06.550739 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4490d2-b411-4e37-85af-03ba6f2e3212 None admin] RBAC: Adding query filter params (role.id=580aa7dbb38d4cff8f9d1bb8983f99ef, user.id=3e1d2954e5cd4a9c94939e2e0979bb08, scope.system=all) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:57:06.550925 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984548bc-b0d8-44df-a1e3-b70433ccb295 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:06.551028 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4490d2-b411-4e37-85af-03ba6f2e3212 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:06.552457 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4490d2-b411-4e37-85af-03ba6f2e3212 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:06.558928 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 144/282] 10.222.0.62 () {56 vars in 1308 bytes} [Wed Aug 24 05:57:06 2022] GET /identity/v3/role_assignments?user.id=3e1d2954e5cd4a9c94939e2e0979bb08&scope.system=all&role.id=580aa7dbb38d4cff8f9d1bb8983f99ef => generated 226 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:57:06.562764 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 139/283] 10.222.0.62 () {56 vars in 1101 bytes} [Wed Aug 24 05:57:06 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:57:06.568803 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4490d2-b411-4e37-85af-03ba6f2e3212 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:06.667556 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871d188f-8d01-4e77-9253-3a426ee2da39 None None] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:06.667895 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871d188f-8d01-4e77-9253-3a426ee2da39 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:06.668107 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871d188f-8d01-4e77-9253-3a426ee2da39 None None] PATH_INFO: `/` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:06.669482 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 140/284] 10.222.0.62 () {54 vars in 927 bytes} [Wed Aug 24 05:57:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:57:06.676164 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8294d6-4366-4047-a48e-a00f1065b4fc None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:06.676440 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8294d6-4366-4047-a48e-a00f1065b4fc None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:06.676816 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8294d6-4366-4047-a48e-a00f1065b4fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:06.684780 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:06.698679 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4490d2-b411-4e37-85af-03ba6f2e3212 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:06.707892 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-8d8294d6-4366-4047-a48e-a00f1065b4fc None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:57:06.709817 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-8d8294d6-4366-4047-a48e-a00f1065b4fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:57:06.757053 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cf425a-99d1-420b-b345-a40632aa45ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:06.757623 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cf425a-99d1-420b-b345-a40632aa45ac None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:06.757927 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cf425a-99d1-420b-b345-a40632aa45ac None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:06.758203 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cf425a-99d1-420b-b345-a40632aa45ac None admin] PATH_INFO: `/v3/users/54248dd26f694bbe851c6f0963a9e349` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:06.759790 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75cf425a-99d1-420b-b345-a40632aa45ac None admin] RBAC: Authorizing `identity:get_user(user_id=54248dd26f694bbe851c6f0963a9e349)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:06.761264 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75cf425a-99d1-420b-b345-a40632aa45ac None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:06.763868 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 145/285] 10.222.0.62 () {56 vars in 1177 bytes} [Wed Aug 24 05:57:06 2022] GET /identity/v3/users/54248dd26f694bbe851c6f0963a9e349 => generated 297 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:57:06.769407 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cf425a-99d1-420b-b345-a40632aa45ac None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:06.781986 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:06.808584 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 141/286] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Aug 24 05:57:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 05:57:06.916652 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cf425a-99d1-420b-b345-a40632aa45ac None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:06.969743 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987be053-c3c3-4ade-8275-977171b81ee0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:06.970378 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987be053-c3c3-4ade-8275-977171b81ee0 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:06.970682 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987be053-c3c3-4ade-8275-977171b81ee0 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:06.970953 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987be053-c3c3-4ade-8275-977171b81ee0 None admin] PATH_INFO: `/v3/projects/9391bbe626b44799942ef25c41bc33ad` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:06.972082 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987be053-c3c3-4ade-8275-977171b81ee0 None admin] RBAC: Authorizing `identity:get_project(project_id=9391bbe626b44799942ef25c41bc33ad)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:06.973530 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987be053-c3c3-4ade-8275-977171b81ee0 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:06.975317 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 146/287] 10.222.0.62 () {56 vars in 1186 bytes} [Wed Aug 24 05:57:06 2022] GET /identity/v3/projects/9391bbe626b44799942ef25c41bc33ad => generated 302 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:57:06.980971 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8294d6-4366-4047-a48e-a00f1065b4fc None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:07.097535 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8294d6-4366-4047-a48e-a00f1065b4fc None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:07.127756 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:07.132795 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7101ed-e0d7-4071-a861-7917bc6abeda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:07.133469 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7101ed-e0d7-4071-a861-7917bc6abeda None admin] REQUEST_METHOD: `PUT` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:07.133750 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7101ed-e0d7-4071-a861-7917bc6abeda None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:07.134027 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7101ed-e0d7-4071-a861-7917bc6abeda None admin] PATH_INFO: `/v3/projects/9391bbe626b44799942ef25c41bc33ad/users/54248dd26f694bbe851c6f0963a9e349/roles/e2f40474c6de4f5fadfa7efbe4087971` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:07.139907 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7101ed-e0d7-4071-a861-7917bc6abeda None admin] RBAC: Authorizing `identity:create_grant(project_id=9391bbe626b44799942ef25c41bc33ad, user_id=54248dd26f694bbe851c6f0963a9e349, role_id=e2f40474c6de4f5fadfa7efbe4087971)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:07.139907 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7101ed-e0d7-4071-a861-7917bc6abeda None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:07.176713 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987be053-c3c3-4ade-8275-977171b81ee0 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:07.238245 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 142/288] 10.222.0.62 () {58 vars in 1439 bytes} [Wed Aug 24 05:57:06 2022] PUT /identity/v3/projects/9391bbe626b44799942ef25c41bc33ad/users/54248dd26f694bbe851c6f0963a9e349/roles/e2f40474c6de4f5fadfa7efbe4087971 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:57:07.264890 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:07.285017 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987be053-c3c3-4ade-8275-977171b81ee0 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:07.330661 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980a5f95-7083-44e9-8025-96338978a3dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:07.331317 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980a5f95-7083-44e9-8025-96338978a3dd None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:07.331590 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980a5f95-7083-44e9-8025-96338978a3dd None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:07.331853 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980a5f95-7083-44e9-8025-96338978a3dd None admin] PATH_INFO: `/v3/roles/member` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:07.342714 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980a5f95-7083-44e9-8025-96338978a3dd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:57:07.348007 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-980a5f95-7083-44e9-8025-96338978a3dd None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 05:57:07.348007 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:57:07.348007 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:57:07.348007 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:57:07.348007 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:57:07.348007 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:57:07.348007 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:57:07.348007 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:57:07.348007 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:57:07.348007 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:57:07.348007 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:57:07.348007 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:57:07.348007 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:57:07.348007 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:57:07.348007 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:57:07.348007 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:57:07.348911 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:57:07.348911 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:57:07.348911 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:57:07.348911 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:57:07.348911 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:57:07.348911 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:57:07.348911 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:57:07.348911 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:57:07.348911 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:57:07.348911 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:57:07.348911 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:57:07.348911 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 24 05:57:07.348911 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:57:07.348911 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:57:07.348911 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:57:07.348911 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:57:07.349716 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:57:07.349716 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:57:07.349716 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:57:07.349716 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:57:07.349716 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:57:07.349716 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:57:07.349716 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:57:07.349716 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:57:07.349716 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:57:07.349716 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:57:07.349716 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 05:57:07.349716 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 24 05:57:07.349716 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 147/289] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Aug 24 05:57:07 2022] GET /identity/v3/roles/member => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 05:57:07.355269 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7101ed-e0d7-4071-a861-7917bc6abeda None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:07.497071 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7101ed-e0d7-4071-a861-7917bc6abeda None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:07.534088 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69b1789-9079-4a2d-9517-a0514cbf2fae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:07.534696 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69b1789-9079-4a2d-9517-a0514cbf2fae None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:07.534972 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69b1789-9079-4a2d-9517-a0514cbf2fae None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:07.535320 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69b1789-9079-4a2d-9517-a0514cbf2fae None admin] PATH_INFO: `/v3/roles` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:07.536016 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69b1789-9079-4a2d-9517-a0514cbf2fae None admin] RBAC: Adding query filter params (name=member) {{(pid=81851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:57:07.536317 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69b1789-9079-4a2d-9517-a0514cbf2fae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:07.537760 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69b1789-9079-4a2d-9517-a0514cbf2fae None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:07.546891 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 143/290] 10.222.0.62 () {56 vars in 1101 bytes} [Wed Aug 24 05:57:07 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:57:07.553796 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980a5f95-7083-44e9-8025-96338978a3dd None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:07.658748 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980a5f95-7083-44e9-8025-96338978a3dd None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:07.692785 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9488f8-9374-4a18-a6e5-516c308e86a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:07.693423 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9488f8-9374-4a18-a6e5-516c308e86a3 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:07.693699 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9488f8-9374-4a18-a6e5-516c308e86a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:07.693969 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9488f8-9374-4a18-a6e5-516c308e86a3 None admin] PATH_INFO: `/v3/users/6e01f2a8b34743a9b2a7cc1d67be9890` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:07.695096 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9488f8-9374-4a18-a6e5-516c308e86a3 None admin] RBAC: Authorizing `identity:get_user(user_id=6e01f2a8b34743a9b2a7cc1d67be9890)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:07.696566 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9488f8-9374-4a18-a6e5-516c308e86a3 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:07.698386 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 148/291] 10.222.0.62 () {56 vars in 1177 bytes} [Wed Aug 24 05:57:07 2022] GET /identity/v3/users/6e01f2a8b34743a9b2a7cc1d67be9890 => generated 297 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:57:07.704807 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69b1789-9079-4a2d-9517-a0514cbf2fae None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:07.789036 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:07.847020 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69b1789-9079-4a2d-9517-a0514cbf2fae None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:07.896359 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883b4094-91b6-4795-a378-bfaf6892848c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:07.896991 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883b4094-91b6-4795-a378-bfaf6892848c None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:07.897268 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883b4094-91b6-4795-a378-bfaf6892848c None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:07.897538 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883b4094-91b6-4795-a378-bfaf6892848c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:07.995877 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883b4094-91b6-4795-a378-bfaf6892848c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:07.997384 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883b4094-91b6-4795-a378-bfaf6892848c None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:08.056187 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:08.119570 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 144/292] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Aug 24 05:57:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 05:57:08.125008 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9488f8-9374-4a18-a6e5-516c308e86a3 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:08.134958 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:08.260588 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9488f8-9374-4a18-a6e5-516c308e86a3 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:08.311591 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c33ea4-cb67-4ee4-8282-f7781e57e8ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:08.312234 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c33ea4-cb67-4ee4-8282-f7781e57e8ca None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:08.312522 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c33ea4-cb67-4ee4-8282-f7781e57e8ca None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:08.312824 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c33ea4-cb67-4ee4-8282-f7781e57e8ca None admin] PATH_INFO: `/v3/projects/9391bbe626b44799942ef25c41bc33ad` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:08.313956 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c33ea4-cb67-4ee4-8282-f7781e57e8ca None admin] RBAC: Authorizing `identity:get_project(project_id=9391bbe626b44799942ef25c41bc33ad)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:08.315912 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c33ea4-cb67-4ee4-8282-f7781e57e8ca None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:08.317572 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 149/293] 10.222.0.62 () {56 vars in 1186 bytes} [Wed Aug 24 05:57:08 2022] GET /identity/v3/projects/9391bbe626b44799942ef25c41bc33ad => generated 302 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:57:08.323627 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883b4094-91b6-4795-a378-bfaf6892848c None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:08.434277 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883b4094-91b6-4795-a378-bfaf6892848c None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:08.475860 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6073c2b-ad56-4f96-b61b-2ecf3a7f9517 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:08.476477 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6073c2b-ad56-4f96-b61b-2ecf3a7f9517 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:08.476777 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6073c2b-ad56-4f96-b61b-2ecf3a7f9517 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:08.477049 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6073c2b-ad56-4f96-b61b-2ecf3a7f9517 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:08.477774 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6073c2b-ad56-4f96-b61b-2ecf3a7f9517 None admin] RBAC: Adding query filter params (role.id=580aa7dbb38d4cff8f9d1bb8983f99ef, scope.project.id=9391bbe626b44799942ef25c41bc33ad, user.id=6e01f2a8b34743a9b2a7cc1d67be9890) {{(pid=81851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:57:08.478080 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6073c2b-ad56-4f96-b61b-2ecf3a7f9517 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:08.479508 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6073c2b-ad56-4f96-b61b-2ecf3a7f9517 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:08.486004 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 145/294] 10.222.0.62 () {56 vars in 1374 bytes} [Wed Aug 24 05:57:08 2022] GET /identity/v3/role_assignments?user.id=6e01f2a8b34743a9b2a7cc1d67be9890&scope.project.id=9391bbe626b44799942ef25c41bc33ad&role.id=580aa7dbb38d4cff8f9d1bb8983f99ef => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:57:08.729226 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e308db-b181-4229-98bc-1e64ce1a0364 None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:08.729552 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e308db-b181-4229-98bc-1e64ce1a0364 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:08.729777 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e308db-b181-4229-98bc-1e64ce1a0364 None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:08.731277 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 150/295] 10.222.0.62 () {54 vars in 927 bytes} [Wed Aug 24 05:57:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:57:08.736382 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b41f62b-132f-4f5d-b9e1-15e7302cfda7 None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:08.736687 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b41f62b-132f-4f5d-b9e1-15e7302cfda7 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:08.736964 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b41f62b-132f-4f5d-b9e1-15e7302cfda7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:08.756692 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-1b41f62b-132f-4f5d-b9e1-15e7302cfda7 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:57:08.757876 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-1b41f62b-132f-4f5d-b9e1-15e7302cfda7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:57:08.773099 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:08.835234 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 146/296] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Aug 24 05:57:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 05:57:08.991068 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc58a9a-dfd0-4e01-9e78-d6aa102bb5b2 None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:08.991270 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc58a9a-dfd0-4e01-9e78-d6aa102bb5b2 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:08.991545 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc58a9a-dfd0-4e01-9e78-d6aa102bb5b2 None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:08.993167 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 151/297] 10.222.0.62 () {54 vars in 927 bytes} [Wed Aug 24 05:57:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:57:08.998199 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e562a1-49b9-4533-862d-6e68fed4819b None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:08.998484 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e562a1-49b9-4533-862d-6e68fed4819b None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:08.998760 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e562a1-49b9-4533-862d-6e68fed4819b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:09.024186 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-93e562a1-49b9-4533-862d-6e68fed4819b None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:57:09.025399 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-93e562a1-49b9-4533-862d-6e68fed4819b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:57:09.097881 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 147/298] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Aug 24 05:57:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 05:57:09.121595 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc58a9a-dfd0-4e01-9e78-d6aa102bb5b2 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:09.200824 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:09.220140 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86aae67d-6828-4f36-953b-3d4a7956de6a None None] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:09.220574 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86aae67d-6828-4f36-953b-3d4a7956de6a None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:09.220851 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86aae67d-6828-4f36-953b-3d4a7956de6a None None] PATH_INFO: `/` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:09.222376 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 148/299] 10.222.0.62 () {54 vars in 927 bytes} [Wed Aug 24 05:57:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:57:09.230745 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffed5d2e-2e56-4404-a3ec-0bacf7de5e2e None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:09.231099 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffed5d2e-2e56-4404-a3ec-0bacf7de5e2e None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:09.231460 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffed5d2e-2e56-4404-a3ec-0bacf7de5e2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:09.235597 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc58a9a-dfd0-4e01-9e78-d6aa102bb5b2 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:09.249719 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-ffed5d2e-2e56-4404-a3ec-0bacf7de5e2e None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:57:09.250961 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-ffed5d2e-2e56-4404-a3ec-0bacf7de5e2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:57:09.277264 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185c7f35-5d29-4b7a-898c-2a8f96032713 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:09.278456 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185c7f35-5d29-4b7a-898c-2a8f96032713 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:09.278915 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185c7f35-5d29-4b7a-898c-2a8f96032713 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:09.279513 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185c7f35-5d29-4b7a-898c-2a8f96032713 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:09.292619 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185c7f35-5d29-4b7a-898c-2a8f96032713 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:57:09.300236 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-185c7f35-5d29-4b7a-898c-2a8f96032713 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 05:57:09.300236 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:57:09.300236 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:57:09.300236 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:57:09.300236 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:57:09.300236 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:57:09.300236 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:57:09.300236 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:57:09.300236 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:57:09.300236 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:57:09.300236 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:57:09.300236 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:57:09.300236 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:57:09.300236 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:57:09.300236 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:57:09.300236 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:57:09.301140 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:57:09.301140 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:57:09.301140 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:57:09.301140 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:57:09.301140 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:57:09.301140 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:57:09.301140 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:57:09.301140 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:57:09.301140 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:57:09.301140 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:57:09.301140 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:57:09.301140 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 24 05:57:09.301140 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:57:09.301140 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:57:09.301140 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:57:09.301140 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:57:09.301934 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:57:09.301934 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:57:09.301934 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:57:09.301934 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:57:09.301934 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:57:09.301934 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:57:09.301934 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:57:09.301934 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:57:09.301934 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:57:09.301934 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:57:09.301934 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 05:57:09.301934 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 24 05:57:09.304271 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 152/300] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Aug 24 05:57:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:57:09.309819 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185c7f35-5d29-4b7a-898c-2a8f96032713 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:09.328467 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 149/301] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Aug 24 05:57:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 05:57:09.392934 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffed5d2e-2e56-4404-a3ec-0bacf7de5e2e None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:09.408867 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:09.429941 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185c7f35-5d29-4b7a-898c-2a8f96032713 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:09.465250 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d431d6-3757-4b21-b128-9939f260f9b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:09.465959 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d431d6-3757-4b21-b128-9939f260f9b2 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:09.466242 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d431d6-3757-4b21-b128-9939f260f9b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:09.466520 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d431d6-3757-4b21-b128-9939f260f9b2 None admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:09.467176 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d431d6-3757-4b21-b128-9939f260f9b2 None admin] RBAC: Adding query filter params (name=member) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:57:09.467477 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d431d6-3757-4b21-b128-9939f260f9b2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:09.469011 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d431d6-3757-4b21-b128-9939f260f9b2 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:09.471152 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:09.479306 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 153/302] 10.222.0.62 () {56 vars in 1101 bytes} [Wed Aug 24 05:57:09 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:57:09.484599 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d431d6-3757-4b21-b128-9939f260f9b2 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:09.503569 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffed5d2e-2e56-4404-a3ec-0bacf7de5e2e None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:09.542188 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d2b337-d882-4be5-a5ab-28a51a7be350 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:09.543086 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d2b337-d882-4be5-a5ab-28a51a7be350 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:09.543370 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d2b337-d882-4be5-a5ab-28a51a7be350 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:09.544066 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d2b337-d882-4be5-a5ab-28a51a7be350 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:09.558396 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d2b337-d882-4be5-a5ab-28a51a7be350 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:57:09.564554 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application [None req-93d2b337-d882-4be5-a5ab-28a51a7be350 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 05:57:09.564554 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:57:09.564554 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:57:09.564554 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:57:09.564554 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:57:09.564554 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:57:09.564554 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:57:09.564554 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:57:09.564554 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:57:09.564554 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:57:09.564554 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:57:09.564554 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:57:09.564554 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:57:09.564554 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:57:09.564554 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:57:09.564554 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:57:09.565976 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:57:09.565976 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:57:09.565976 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:57:09.565976 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:57:09.565976 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:57:09.565976 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:57:09.565976 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:57:09.565976 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:57:09.565976 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:57:09.565976 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:57:09.565976 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:57:09.565976 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application with Lock( Aug 24 05:57:09.565976 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:57:09.565976 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:57:09.565976 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:57:09.565976 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:57:09.567162 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:57:09.567162 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:57:09.567162 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:57:09.567162 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:57:09.567162 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:57:09.567162 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:57:09.567162 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:57:09.567162 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:57:09.567162 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:57:09.567162 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:57:09.567162 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 05:57:09.567162 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Aug 24 05:57:09.572529 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 150/303] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Aug 24 05:57:09 2022] GET /identity/v3/roles/reader => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 05:57:09.572529 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d2b337-d882-4be5-a5ab-28a51a7be350 None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:09.597662 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d431d6-3757-4b21-b128-9939f260f9b2 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:09.632140 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18172ef-fc92-4d20-a238-12b89ad14a51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:09.632815 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18172ef-fc92-4d20-a238-12b89ad14a51 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:09.633094 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18172ef-fc92-4d20-a238-12b89ad14a51 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:09.633368 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18172ef-fc92-4d20-a238-12b89ad14a51 None admin] PATH_INFO: `/v3/users/3e1d2954e5cd4a9c94939e2e0979bb08` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:09.634486 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18172ef-fc92-4d20-a238-12b89ad14a51 None admin] RBAC: Authorizing `identity:get_user(user_id=3e1d2954e5cd4a9c94939e2e0979bb08)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:09.635917 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18172ef-fc92-4d20-a238-12b89ad14a51 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:09.642093 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 154/304] 10.222.0.62 () {56 vars in 1177 bytes} [Wed Aug 24 05:57:09 2022] GET /identity/v3/users/3e1d2954e5cd4a9c94939e2e0979bb08 => generated 293 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:57:09.642093 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18172ef-fc92-4d20-a238-12b89ad14a51 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:09.654116 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:09.688309 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d2b337-d882-4be5-a5ab-28a51a7be350 None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:09.703890 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:09.723767 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19326579-0626-453e-9da6-b16fd6eefad4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:09.724424 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19326579-0626-453e-9da6-b16fd6eefad4 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:09.724734 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19326579-0626-453e-9da6-b16fd6eefad4 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:09.725002 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19326579-0626-453e-9da6-b16fd6eefad4 None admin] PATH_INFO: `/v3/roles` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:09.725658 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19326579-0626-453e-9da6-b16fd6eefad4 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:57:09.725957 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19326579-0626-453e-9da6-b16fd6eefad4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:09.727324 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19326579-0626-453e-9da6-b16fd6eefad4 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:09.735594 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 151/305] 10.222.0.62 () {56 vars in 1101 bytes} [Wed Aug 24 05:57:09 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:57:09.736425 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19326579-0626-453e-9da6-b16fd6eefad4 None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:09.747025 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18172ef-fc92-4d20-a238-12b89ad14a51 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:09.781332 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e874672-ee1f-4d7f-9f5a-f130d8e628e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:09.781975 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e874672-ee1f-4d7f-9f5a-f130d8e628e1 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:09.782257 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e874672-ee1f-4d7f-9f5a-f130d8e628e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:09.782554 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e874672-ee1f-4d7f-9f5a-f130d8e628e1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:09.783667 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e874672-ee1f-4d7f-9f5a-f130d8e628e1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:09.785143 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e874672-ee1f-4d7f-9f5a-f130d8e628e1 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:09.786718 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 155/306] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Aug 24 05:57:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:57:09.787842 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3000c1-d8b9-48b5-930c-3d7e198f7bad None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:09.788120 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3000c1-d8b9-48b5-930c-3d7e198f7bad None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:09.788391 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3000c1-d8b9-48b5-930c-3d7e198f7bad None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:09.793611 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 156/307] 10.222.0.62 () {54 vars in 927 bytes} [Wed Aug 24 05:57:09 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:57:09.794379 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3000c1-d8b9-48b5-930c-3d7e198f7bad None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:09.837300 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19326579-0626-453e-9da6-b16fd6eefad4 None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:09.864911 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:09.870375 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618de08e-6ccd-4c52-bc05-08d24999fe33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:09.871018 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618de08e-6ccd-4c52-bc05-08d24999fe33 None admin] REQUEST_METHOD: `PUT` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:09.871347 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618de08e-6ccd-4c52-bc05-08d24999fe33 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:09.871594 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618de08e-6ccd-4c52-bc05-08d24999fe33 None admin] PATH_INFO: `/v3/system/users/3e1d2954e5cd4a9c94939e2e0979bb08/roles/580aa7dbb38d4cff8f9d1bb8983f99ef` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:09.873001 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618de08e-6ccd-4c52-bc05-08d24999fe33 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=3e1d2954e5cd4a9c94939e2e0979bb08, role_id=580aa7dbb38d4cff8f9d1bb8983f99ef)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:09.874385 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618de08e-6ccd-4c52-bc05-08d24999fe33 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:09.893773 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3000c1-d8b9-48b5-930c-3d7e198f7bad None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:09.921931 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:09.926606 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d34348-6781-488e-81c9-2831c0688db0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:09.927223 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d34348-6781-488e-81c9-2831c0688db0 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:09.927546 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d34348-6781-488e-81c9-2831c0688db0 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:09.927823 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d34348-6781-488e-81c9-2831c0688db0 None admin] PATH_INFO: `/v3/users/5a808a34e0d24205a4b92f987a85eacf` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:09.940863 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d34348-6781-488e-81c9-2831c0688db0 None admin] RBAC: Authorizing `identity:get_user(user_id=5a808a34e0d24205a4b92f987a85eacf)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:09.942307 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d34348-6781-488e-81c9-2831c0688db0 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:09.943884 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 157/308] 10.222.0.62 () {56 vars in 1177 bytes} [Wed Aug 24 05:57:09 2022] GET /identity/v3/users/5a808a34e0d24205a4b92f987a85eacf => generated 293 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:57:09.944648 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d34348-6781-488e-81c9-2831c0688db0 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:09.955795 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 152/309] 10.222.0.62 () {58 vars in 1334 bytes} [Wed Aug 24 05:57:09 2022] PUT /identity/v3/system/users/3e1d2954e5cd4a9c94939e2e0979bb08/roles/580aa7dbb38d4cff8f9d1bb8983f99ef => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:57:09.968767 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d1f226-1af4-449a-9e65-9ab861a45c7f None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:09.968767 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d1f226-1af4-449a-9e65-9ab861a45c7f None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:09.968767 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d1f226-1af4-449a-9e65-9ab861a45c7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:09.976439 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-42d1f226-1af4-449a-9e65-9ab861a45c7f None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:57:09.977695 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-42d1f226-1af4-449a-9e65-9ab861a45c7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:57:10.056726 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 153/310] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Aug 24 05:57:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 05:57:10.056726 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d1f226-1af4-449a-9e65-9ab861a45c7f None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:10.075694 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d34348-6781-488e-81c9-2831c0688db0 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:10.126188 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0762e4bf-1e6e-42da-a0d2-6ffeb8bc5569 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:10.127367 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0762e4bf-1e6e-42da-a0d2-6ffeb8bc5569 None admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:10.127646 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0762e4bf-1e6e-42da-a0d2-6ffeb8bc5569 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:10.127916 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0762e4bf-1e6e-42da-a0d2-6ffeb8bc5569 None admin] PATH_INFO: `/v3/groups` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:10.129133 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0762e4bf-1e6e-42da-a0d2-6ffeb8bc5569 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:10.130855 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0762e4bf-1e6e-42da-a0d2-6ffeb8bc5569 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:10.177731 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d1f226-1af4-449a-9e65-9ab861a45c7f None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:10.205593 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 158/311] 10.222.0.62 () {60 vars in 1134 bytes} [Wed Aug 24 05:57:09 2022] POST /identity/v3/groups => generated 229 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:57:10.220214 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:10.234076 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c17ec6-a69a-4db8-b2eb-52589458958d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:10.234756 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c17ec6-a69a-4db8-b2eb-52589458958d None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:10.235037 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c17ec6-a69a-4db8-b2eb-52589458958d None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:10.235307 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c17ec6-a69a-4db8-b2eb-52589458958d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:10.236613 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c17ec6-a69a-4db8-b2eb-52589458958d None admin] RBAC: Adding query filter params (role.id=e2f40474c6de4f5fadfa7efbe4087971, user.id=5a808a34e0d24205a4b92f987a85eacf, scope.system=all) {{(pid=81851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:57:10.236922 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c17ec6-a69a-4db8-b2eb-52589458958d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:10.238394 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c17ec6-a69a-4db8-b2eb-52589458958d None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:10.244866 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 154/312] 10.222.0.62 () {56 vars in 1308 bytes} [Wed Aug 24 05:57:10 2022] GET /identity/v3/role_assignments?user.id=5a808a34e0d24205a4b92f987a85eacf&scope.system=all&role.id=e2f40474c6de4f5fadfa7efbe4087971 => generated 226 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:57:10.334819 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0762e4bf-1e6e-42da-a0d2-6ffeb8bc5569 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:10.395454 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:10.451401 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0762e4bf-1e6e-42da-a0d2-6ffeb8bc5569 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:10.490538 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f007ad75-9d11-4394-b87c-947d2e40aaf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:10.491199 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f007ad75-9d11-4394-b87c-947d2e40aaf2 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:10.491482 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f007ad75-9d11-4394-b87c-947d2e40aaf2 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:10.491761 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f007ad75-9d11-4394-b87c-947d2e40aaf2 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:10.502959 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f007ad75-9d11-4394-b87c-947d2e40aaf2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:57:10.508267 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-f007ad75-9d11-4394-b87c-947d2e40aaf2 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 05:57:10.508267 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:57:10.508267 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:57:10.508267 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:57:10.508267 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:57:10.508267 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:57:10.508267 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:57:10.508267 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:57:10.508267 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:57:10.508267 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:57:10.508267 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:57:10.508267 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:57:10.508267 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:57:10.508267 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:57:10.508267 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:57:10.508267 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:57:10.509155 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:57:10.509155 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:57:10.509155 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:57:10.509155 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:57:10.509155 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:57:10.509155 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:57:10.509155 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:57:10.509155 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:57:10.509155 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:57:10.509155 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:57:10.509155 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:57:10.509155 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 24 05:57:10.509155 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:57:10.509155 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:57:10.509155 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:57:10.509155 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:57:10.510118 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:57:10.510118 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:57:10.510118 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:57:10.510118 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:57:10.510118 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:57:10.510118 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:57:10.510118 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:57:10.510118 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:57:10.510118 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:57:10.510118 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:57:10.510118 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 05:57:10.510118 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 24 05:57:10.510118 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 159/313] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Aug 24 05:57:10 2022] GET /identity/v3/roles/reader => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:57:10.515148 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c17ec6-a69a-4db8-b2eb-52589458958d None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:10.622580 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c17ec6-a69a-4db8-b2eb-52589458958d None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:10.659268 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88faf7e-4e42-4ec2-b489-a77a64933db1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:10.660167 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88faf7e-4e42-4ec2-b489-a77a64933db1 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:10.660681 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88faf7e-4e42-4ec2-b489-a77a64933db1 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:10.661124 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88faf7e-4e42-4ec2-b489-a77a64933db1 None admin] PATH_INFO: `/v3/roles` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:10.661949 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88faf7e-4e42-4ec2-b489-a77a64933db1 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:57:10.662472 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88faf7e-4e42-4ec2-b489-a77a64933db1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:10.664054 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88faf7e-4e42-4ec2-b489-a77a64933db1 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:10.673281 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 155/314] 10.222.0.62 () {56 vars in 1101 bytes} [Wed Aug 24 05:57:10 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:57:10.684515 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f007ad75-9d11-4394-b87c-947d2e40aaf2 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:10.793476 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f007ad75-9d11-4394-b87c-947d2e40aaf2 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:10.806390 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:10.829502 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743d6bf2-4ffa-4f87-8b80-069cdc788c7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:10.830730 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743d6bf2-4ffa-4f87-8b80-069cdc788c7e None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:10.831011 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743d6bf2-4ffa-4f87-8b80-069cdc788c7e None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:10.831281 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743d6bf2-4ffa-4f87-8b80-069cdc788c7e None admin] PATH_INFO: `/v3/users/54248dd26f694bbe851c6f0963a9e349` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:10.832407 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743d6bf2-4ffa-4f87-8b80-069cdc788c7e None admin] RBAC: Authorizing `identity:get_user(user_id=54248dd26f694bbe851c6f0963a9e349)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:10.833861 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743d6bf2-4ffa-4f87-8b80-069cdc788c7e None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:10.835745 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 160/315] 10.222.0.62 () {56 vars in 1177 bytes} [Wed Aug 24 05:57:10 2022] GET /identity/v3/users/54248dd26f694bbe851c6f0963a9e349 => generated 297 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:57:10.841128 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88faf7e-4e42-4ec2-b489-a77a64933db1 None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:10.897689 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:10.948841 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88faf7e-4e42-4ec2-b489-a77a64933db1 None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:10.983423 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee02415-1b0b-4d2f-a1f8-6ddcc4d6589a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:10.984040 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee02415-1b0b-4d2f-a1f8-6ddcc4d6589a None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:10.984322 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee02415-1b0b-4d2f-a1f8-6ddcc4d6589a None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:10.984631 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee02415-1b0b-4d2f-a1f8-6ddcc4d6589a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:11.054373 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee02415-1b0b-4d2f-a1f8-6ddcc4d6589a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:11.055852 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee02415-1b0b-4d2f-a1f8-6ddcc4d6589a None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:11.109690 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:11.156526 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 156/316] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Aug 24 05:57:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 05:57:11.162593 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743d6bf2-4ffa-4f87-8b80-069cdc788c7e None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:11.267263 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743d6bf2-4ffa-4f87-8b80-069cdc788c7e None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:11.302329 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb4db58-bc38-4366-9c25-d09b5c52bbbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:11.302968 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb4db58-bc38-4366-9c25-d09b5c52bbbf None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:11.303247 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb4db58-bc38-4366-9c25-d09b5c52bbbf None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:11.303520 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb4db58-bc38-4366-9c25-d09b5c52bbbf None admin] PATH_INFO: `/v3/projects/9391bbe626b44799942ef25c41bc33ad` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:11.304715 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb4db58-bc38-4366-9c25-d09b5c52bbbf None admin] RBAC: Authorizing `identity:get_project(project_id=9391bbe626b44799942ef25c41bc33ad)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:11.306147 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb4db58-bc38-4366-9c25-d09b5c52bbbf None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:11.308153 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 161/317] 10.222.0.62 () {56 vars in 1186 bytes} [Wed Aug 24 05:57:11 2022] GET /identity/v3/projects/9391bbe626b44799942ef25c41bc33ad => generated 302 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:57:11.314995 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee02415-1b0b-4d2f-a1f8-6ddcc4d6589a None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:11.417796 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee02415-1b0b-4d2f-a1f8-6ddcc4d6589a None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:11.452775 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00cdede-048b-4ca5-be56-33cf1cc258ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:11.453385 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00cdede-048b-4ca5-be56-33cf1cc258ed None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:11.453664 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00cdede-048b-4ca5-be56-33cf1cc258ed None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:11.453941 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00cdede-048b-4ca5-be56-33cf1cc258ed None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:11.454703 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00cdede-048b-4ca5-be56-33cf1cc258ed None admin] RBAC: Adding query filter params (role.id=e2f40474c6de4f5fadfa7efbe4087971, scope.project.id=9391bbe626b44799942ef25c41bc33ad, user.id=54248dd26f694bbe851c6f0963a9e349) {{(pid=81851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:57:11.455521 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00cdede-048b-4ca5-be56-33cf1cc258ed None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:11.457052 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00cdede-048b-4ca5-be56-33cf1cc258ed None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:11.462933 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 157/318] 10.222.0.62 () {56 vars in 1374 bytes} [Wed Aug 24 05:57:11 2022] GET /identity/v3/role_assignments?user.id=54248dd26f694bbe851c6f0963a9e349&scope.project.id=9391bbe626b44799942ef25c41bc33ad&role.id=e2f40474c6de4f5fadfa7efbe4087971 => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:57:11.905784 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9507aa03-cf3e-4e10-a087-66320dbeb606 None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:11.906148 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9507aa03-cf3e-4e10-a087-66320dbeb606 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:11.906348 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9507aa03-cf3e-4e10-a087-66320dbeb606 None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:11.908416 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 162/319] 10.222.0.62 () {54 vars in 927 bytes} [Wed Aug 24 05:57:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:57:11.913482 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfaedf2b-3482-458e-b922-b1ef8115ff0d None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:11.913929 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfaedf2b-3482-458e-b922-b1ef8115ff0d None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:11.914095 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfaedf2b-3482-458e-b922-b1ef8115ff0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:11.932548 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-bfaedf2b-3482-458e-b922-b1ef8115ff0d None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:57:11.933732 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-bfaedf2b-3482-458e-b922-b1ef8115ff0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:57:11.944378 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:12.002055 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 158/320] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Aug 24 05:57:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 05:57:12.156949 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a255d70-4301-41ef-bf0a-01e1a8aa7e30 None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:12.157260 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a255d70-4301-41ef-bf0a-01e1a8aa7e30 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:12.157492 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a255d70-4301-41ef-bf0a-01e1a8aa7e30 None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:12.159015 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 163/321] 10.222.0.62 () {54 vars in 927 bytes} [Wed Aug 24 05:57:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:57:12.163983 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5320f1-4c37-40b4-afcd-d1e7f0f289ff None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:12.164268 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5320f1-4c37-40b4-afcd-d1e7f0f289ff None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:12.164561 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5320f1-4c37-40b4-afcd-d1e7f0f289ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:12.182728 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-ae5320f1-4c37-40b4-afcd-d1e7f0f289ff None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:57:12.183965 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-ae5320f1-4c37-40b4-afcd-d1e7f0f289ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:57:12.251176 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 159/322] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Aug 24 05:57:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 05:57:12.273731 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a255d70-4301-41ef-bf0a-01e1a8aa7e30 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:12.328089 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:12.332220 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0490969c-3da1-4ad2-a85c-504f456b8018 None None] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:12.332519 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0490969c-3da1-4ad2-a85c-504f456b8018 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:12.332804 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0490969c-3da1-4ad2-a85c-504f456b8018 None None] PATH_INFO: `/` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:12.334141 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 160/323] 10.222.0.62 () {54 vars in 927 bytes} [Wed Aug 24 05:57:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:57:12.339083 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abc6628-dffd-45d3-9c69-c25b29d77e0d None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:12.339366 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abc6628-dffd-45d3-9c69-c25b29d77e0d None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:12.347353 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abc6628-dffd-45d3-9c69-c25b29d77e0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:12.357587 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-1abc6628-dffd-45d3-9c69-c25b29d77e0d None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:57:12.358984 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-1abc6628-dffd-45d3-9c69-c25b29d77e0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:57:12.388314 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a255d70-4301-41ef-bf0a-01e1a8aa7e30 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:12.420941 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678e0d3b-8073-45be-b102-0cb50ca6c052 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:12.421592 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678e0d3b-8073-45be-b102-0cb50ca6c052 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:12.421871 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678e0d3b-8073-45be-b102-0cb50ca6c052 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:12.422141 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678e0d3b-8073-45be-b102-0cb50ca6c052 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:12.432668 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 161/324] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Aug 24 05:57:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 05:57:12.436745 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678e0d3b-8073-45be-b102-0cb50ca6c052 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:57:12.442045 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-678e0d3b-8073-45be-b102-0cb50ca6c052 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 05:57:12.442045 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:57:12.442045 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:57:12.442045 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:57:12.442045 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:57:12.442045 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:57:12.442045 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:57:12.442045 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:57:12.442045 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:57:12.442045 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:57:12.442045 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:57:12.442045 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:57:12.442045 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:57:12.442045 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:57:12.442045 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:57:12.442045 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:57:12.442884 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:57:12.442884 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:57:12.442884 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:57:12.442884 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:57:12.442884 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:57:12.442884 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:57:12.442884 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:57:12.442884 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:57:12.442884 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:57:12.442884 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:57:12.442884 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:57:12.442884 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 24 05:57:12.442884 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:57:12.442884 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:57:12.442884 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:57:12.442884 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:57:12.443688 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:57:12.443688 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:57:12.443688 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:57:12.443688 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:57:12.443688 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:57:12.443688 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:57:12.443688 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:57:12.443688 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:57:12.443688 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:57:12.443688 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:57:12.443688 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 05:57:12.443688 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 24 05:57:12.444881 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 164/325] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Aug 24 05:57:12 2022] GET /identity/v3/roles/member => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:57:12.448766 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abc6628-dffd-45d3-9c69-c25b29d77e0d None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:12.452814 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:12.520821 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678e0d3b-8073-45be-b102-0cb50ca6c052 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:12.552945 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abc6628-dffd-45d3-9c69-c25b29d77e0d None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:12.586699 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0934e17a-6e1a-411d-91cb-a1cf3e91df20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:12.587485 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0934e17a-6e1a-411d-91cb-a1cf3e91df20 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:12.587865 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0934e17a-6e1a-411d-91cb-a1cf3e91df20 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:12.588187 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0934e17a-6e1a-411d-91cb-a1cf3e91df20 None admin] PATH_INFO: `/v3/roles` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:12.588874 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0934e17a-6e1a-411d-91cb-a1cf3e91df20 None admin] RBAC: Adding query filter params (name=member) {{(pid=81851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:57:12.589228 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0934e17a-6e1a-411d-91cb-a1cf3e91df20 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:12.590590 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0934e17a-6e1a-411d-91cb-a1cf3e91df20 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:12.603135 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 162/326] 10.222.0.62 () {56 vars in 1101 bytes} [Wed Aug 24 05:57:12 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:57:12.604311 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0934e17a-6e1a-411d-91cb-a1cf3e91df20 None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:12.630112 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678e0d3b-8073-45be-b102-0cb50ca6c052 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:12.637100 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:12.668173 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6f87e0-a5aa-4760-8a74-8222a12a55b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:12.668835 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6f87e0-a5aa-4760-8a74-8222a12a55b4 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:12.669111 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6f87e0-a5aa-4760-8a74-8222a12a55b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:12.669379 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6f87e0-a5aa-4760-8a74-8222a12a55b4 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:12.671505 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:12.681317 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6f87e0-a5aa-4760-8a74-8222a12a55b4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:57:12.686122 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-bf6f87e0-a5aa-4760-8a74-8222a12a55b4 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 05:57:12.686122 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:57:12.686122 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:57:12.686122 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:57:12.686122 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:57:12.686122 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:57:12.686122 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:57:12.686122 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:57:12.686122 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:57:12.686122 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:57:12.686122 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:57:12.686122 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:57:12.686122 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:57:12.686122 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:57:12.686122 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:57:12.686122 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:57:12.687014 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:57:12.687014 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:57:12.687014 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:57:12.687014 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:57:12.687014 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:57:12.687014 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:57:12.687014 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:57:12.687014 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:57:12.687014 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:57:12.687014 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:57:12.687014 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:57:12.687014 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 24 05:57:12.687014 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:57:12.687014 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:57:12.687014 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:57:12.687014 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:57:12.687790 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:57:12.687790 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:57:12.687790 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:57:12.687790 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:57:12.687790 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:57:12.687790 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:57:12.687790 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:57:12.687790 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:57:12.687790 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:57:12.687790 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:57:12.687790 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 05:57:12.687790 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 24 05:57:12.687790 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 165/327] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Aug 24 05:57:12 2022] GET /identity/v3/roles/reader => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 05:57:12.693101 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6f87e0-a5aa-4760-8a74-8222a12a55b4 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:12.714831 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0934e17a-6e1a-411d-91cb-a1cf3e91df20 None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:12.749247 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee976ef-6c34-4505-8148-56d5aa7b951a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:12.750025 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee976ef-6c34-4505-8148-56d5aa7b951a None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:12.750303 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee976ef-6c34-4505-8148-56d5aa7b951a None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:12.750573 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee976ef-6c34-4505-8148-56d5aa7b951a None admin] PATH_INFO: `/v3/users/3e1d2954e5cd4a9c94939e2e0979bb08` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:12.751725 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee976ef-6c34-4505-8148-56d5aa7b951a None admin] RBAC: Authorizing `identity:get_user(user_id=3e1d2954e5cd4a9c94939e2e0979bb08)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:12.753194 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee976ef-6c34-4505-8148-56d5aa7b951a None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:12.754760 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 163/328] 10.222.0.62 () {56 vars in 1177 bytes} [Wed Aug 24 05:57:12 2022] GET /identity/v3/users/3e1d2954e5cd4a9c94939e2e0979bb08 => generated 293 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:57:12.759231 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee976ef-6c34-4505-8148-56d5aa7b951a None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:12.794696 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6f87e0-a5aa-4760-8a74-8222a12a55b4 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:12.827367 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b29eef-5a5b-4187-89a3-e948cb022368 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:12.827931 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b29eef-5a5b-4187-89a3-e948cb022368 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:12.828208 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b29eef-5a5b-4187-89a3-e948cb022368 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:12.828472 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b29eef-5a5b-4187-89a3-e948cb022368 None admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:12.829151 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b29eef-5a5b-4187-89a3-e948cb022368 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:57:12.829446 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b29eef-5a5b-4187-89a3-e948cb022368 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:12.830817 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b29eef-5a5b-4187-89a3-e948cb022368 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:12.838699 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 166/329] 10.222.0.62 () {56 vars in 1101 bytes} [Wed Aug 24 05:57:12 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:57:12.839469 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b29eef-5a5b-4187-89a3-e948cb022368 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:12.855477 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee976ef-6c34-4505-8148-56d5aa7b951a None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:12.868310 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:12.888656 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44973152-af2e-4cf5-8b4f-9bbbcbe60680 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:12.889322 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44973152-af2e-4cf5-8b4f-9bbbcbe60680 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:12.889603 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44973152-af2e-4cf5-8b4f-9bbbcbe60680 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:12.889871 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44973152-af2e-4cf5-8b4f-9bbbcbe60680 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:12.891002 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44973152-af2e-4cf5-8b4f-9bbbcbe60680 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:12.892428 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44973152-af2e-4cf5-8b4f-9bbbcbe60680 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:12.894033 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 164/330] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Aug 24 05:57:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:57:12.894788 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44973152-af2e-4cf5-8b4f-9bbbcbe60680 None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:12.898861 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:12.938582 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b29eef-5a5b-4187-89a3-e948cb022368 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:12.972979 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0343ab21-f567-4f9a-a6c2-1708e1a3d435 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:12.973584 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0343ab21-f567-4f9a-a6c2-1708e1a3d435 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:12.973867 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0343ab21-f567-4f9a-a6c2-1708e1a3d435 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:12.974140 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0343ab21-f567-4f9a-a6c2-1708e1a3d435 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:12.974848 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0343ab21-f567-4f9a-a6c2-1708e1a3d435 None admin] RBAC: Adding query filter params (role.id=580aa7dbb38d4cff8f9d1bb8983f99ef, user.id=3e1d2954e5cd4a9c94939e2e0979bb08, scope.system=all) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:57:12.975152 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0343ab21-f567-4f9a-a6c2-1708e1a3d435 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:12.976596 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0343ab21-f567-4f9a-a6c2-1708e1a3d435 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:12.988078 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 167/331] 10.222.0.62 () {56 vars in 1308 bytes} [Wed Aug 24 05:57:12 2022] GET /identity/v3/role_assignments?user.id=3e1d2954e5cd4a9c94939e2e0979bb08&scope.system=all&role.id=580aa7dbb38d4cff8f9d1bb8983f99ef => generated 510 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:57:12.988858 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0343ab21-f567-4f9a-a6c2-1708e1a3d435 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:12.996976 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44973152-af2e-4cf5-8b4f-9bbbcbe60680 None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:13.034565 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5351ec-fd02-4dbb-a098-fcc9b9474cc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:13.035248 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5351ec-fd02-4dbb-a098-fcc9b9474cc8 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:13.035528 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5351ec-fd02-4dbb-a098-fcc9b9474cc8 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:13.035797 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5351ec-fd02-4dbb-a098-fcc9b9474cc8 None admin] PATH_INFO: `/v3/users/5a808a34e0d24205a4b92f987a85eacf` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:13.037071 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5351ec-fd02-4dbb-a098-fcc9b9474cc8 None admin] RBAC: Authorizing `identity:get_user(user_id=5a808a34e0d24205a4b92f987a85eacf)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:13.038525 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5351ec-fd02-4dbb-a098-fcc9b9474cc8 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:13.040212 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 165/332] 10.222.0.62 () {56 vars in 1177 bytes} [Wed Aug 24 05:57:12 2022] GET /identity/v3/users/5a808a34e0d24205a4b92f987a85eacf => generated 293 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:57:13.046404 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5351ec-fd02-4dbb-a098-fcc9b9474cc8 None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:13.085425 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:13.103294 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0343ab21-f567-4f9a-a6c2-1708e1a3d435 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:13.128391 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:13.144226 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1861e66f-e9eb-400a-b4b5-ec64f06ce207 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:13.144902 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1861e66f-e9eb-400a-b4b5-ec64f06ce207 None admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:13.145176 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1861e66f-e9eb-400a-b4b5-ec64f06ce207 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:13.145491 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1861e66f-e9eb-400a-b4b5-ec64f06ce207 None admin] PATH_INFO: `/v3/groups` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:13.146133 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1861e66f-e9eb-400a-b4b5-ec64f06ce207 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:13.149917 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1861e66f-e9eb-400a-b4b5-ec64f06ce207 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:13.163909 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5351ec-fd02-4dbb-a098-fcc9b9474cc8 None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:13.199217 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8586f9a4-2aa6-4ba2-a573-ab457b6d2d5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:13.199887 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8586f9a4-2aa6-4ba2-a573-ab457b6d2d5b None admin] REQUEST_METHOD: `PUT` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:13.200163 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8586f9a4-2aa6-4ba2-a573-ab457b6d2d5b None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:13.200431 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8586f9a4-2aa6-4ba2-a573-ab457b6d2d5b None admin] PATH_INFO: `/v3/system/users/5a808a34e0d24205a4b92f987a85eacf/roles/e2f40474c6de4f5fadfa7efbe4087971` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:13.201882 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8586f9a4-2aa6-4ba2-a573-ab457b6d2d5b None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=5a808a34e0d24205a4b92f987a85eacf, role_id=e2f40474c6de4f5fadfa7efbe4087971)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:13.203251 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8586f9a4-2aa6-4ba2-a573-ab457b6d2d5b None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:13.206499 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 168/333] 10.222.0.62 () {60 vars in 1134 bytes} [Wed Aug 24 05:57:12 2022] POST /identity/v3/groups => generated 226 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:57:13.286810 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 166/334] 10.222.0.62 () {58 vars in 1334 bytes} [Wed Aug 24 05:57:13 2022] PUT /identity/v3/system/users/5a808a34e0d24205a4b92f987a85eacf/roles/e2f40474c6de4f5fadfa7efbe4087971 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 05:57:15.281995 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5befa95-08b1-4d67-b49f-9a6f73f26c06 None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:15.283489 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5befa95-08b1-4d67-b49f-9a6f73f26c06 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:15.283995 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5befa95-08b1-4d67-b49f-9a6f73f26c06 None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:15.285852 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 169/335] 10.222.0.62 () {54 vars in 927 bytes} [Wed Aug 24 05:57:15 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:57:15.290006 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60a78aa-6090-40ad-86ac-72defb0346ef None None] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:15.291730 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c98664-4696-408c-98fd-cd8fa758d7b1 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:15.292211 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c98664-4696-408c-98fd-cd8fa758d7b1 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:15.292718 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c98664-4696-408c-98fd-cd8fa758d7b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:15.296095 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60a78aa-6090-40ad-86ac-72defb0346ef None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:15.296653 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60a78aa-6090-40ad-86ac-72defb0346ef None None] PATH_INFO: `/` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:15.298911 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 167/336] 10.222.0.62 () {54 vars in 927 bytes} [Wed Aug 24 05:57:15 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 05:57:15.316790 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-40c98664-4696-408c-98fd-cd8fa758d7b1 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:57:15.318403 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-40c98664-4696-408c-98fd-cd8fa758d7b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:57:15.323607 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16819ad-a0b1-4133-9a6b-92c8156515cb None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:15.323891 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16819ad-a0b1-4133-9a6b-92c8156515cb None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:15.324162 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16819ad-a0b1-4133-9a6b-92c8156515cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:15.357505 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-c16819ad-a0b1-4133-9a6b-92c8156515cb None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:57:15.358708 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-c16819ad-a0b1-4133-9a6b-92c8156515cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:57:15.439370 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 170/337] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Aug 24 05:57:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 05:57:15.466659 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 168/338] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Aug 24 05:57:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 05:57:15.670651 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acc5fd4-f5f7-43fc-bacd-a9bfd10b20b0 None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:15.670990 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acc5fd4-f5f7-43fc-bacd-a9bfd10b20b0 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:15.671246 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acc5fd4-f5f7-43fc-bacd-a9bfd10b20b0 None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:15.672666 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 171/339] 10.222.0.62 () {54 vars in 927 bytes} [Wed Aug 24 05:57:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:57:15.678530 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa7f3ba-24fd-4b39-8e7e-e9758fd04a32 None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:15.678819 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa7f3ba-24fd-4b39-8e7e-e9758fd04a32 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:15.679090 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa7f3ba-24fd-4b39-8e7e-e9758fd04a32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:15.721112 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-afa7f3ba-24fd-4b39-8e7e-e9758fd04a32 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:57:15.722607 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-afa7f3ba-24fd-4b39-8e7e-e9758fd04a32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:57:15.791643 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:15.801215 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 169/340] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Aug 24 05:57:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 05:57:15.890730 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acc5fd4-f5f7-43fc-bacd-a9bfd10b20b0 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:15.979240 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:15.993854 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa7f3ba-24fd-4b39-8e7e-e9758fd04a32 None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:16.017830 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acc5fd4-f5f7-43fc-bacd-a9bfd10b20b0 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:16.085210 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5265295-afd9-4b77-8882-3e4d9d8b1c1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:16.085867 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5265295-afd9-4b77-8882-3e4d9d8b1c1d None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:16.086144 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5265295-afd9-4b77-8882-3e4d9d8b1c1d None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:16.086412 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5265295-afd9-4b77-8882-3e4d9d8b1c1d None admin] PATH_INFO: `/v3/roles/member` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:16.112525 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5265295-afd9-4b77-8882-3e4d9d8b1c1d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:57:16.117427 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-d5265295-afd9-4b77-8882-3e4d9d8b1c1d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 05:57:16.117427 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:57:16.117427 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:57:16.117427 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:57:16.117427 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:57:16.117427 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:57:16.117427 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:57:16.117427 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:57:16.117427 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:57:16.117427 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:57:16.117427 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:57:16.117427 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:57:16.117427 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:57:16.117427 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:57:16.117427 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:57:16.117427 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:57:16.118324 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:57:16.118324 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:57:16.118324 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:57:16.118324 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:57:16.118324 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:57:16.118324 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:57:16.118324 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:57:16.118324 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:57:16.118324 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:57:16.118324 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:57:16.118324 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:57:16.118324 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 24 05:57:16.118324 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:57:16.118324 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:57:16.118324 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:57:16.118324 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:57:16.119119 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:57:16.119119 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:57:16.119119 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:57:16.119119 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:57:16.119119 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:57:16.119119 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:57:16.119119 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:57:16.119119 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:57:16.119119 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:57:16.119119 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:57:16.119119 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 05:57:16.119119 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 24 05:57:16.128381 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 172/341] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Aug 24 05:57:15 2022] GET /identity/v3/roles/member => generated 84 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:57:16.130803 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5265295-afd9-4b77-8882-3e4d9d8b1c1d None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:16.194655 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa7f3ba-24fd-4b39-8e7e-e9758fd04a32 None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:16.256257 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59589129-2e4d-410e-89cf-8885e8ac686a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:16.256844 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59589129-2e4d-410e-89cf-8885e8ac686a None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:16.257125 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59589129-2e4d-410e-89cf-8885e8ac686a None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:16.257405 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59589129-2e4d-410e-89cf-8885e8ac686a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:16.273518 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59589129-2e4d-410e-89cf-8885e8ac686a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:57:16.278384 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application [None req-59589129-2e4d-410e-89cf-8885e8ac686a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 05:57:16.278384 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:57:16.278384 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:57:16.278384 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:57:16.278384 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:57:16.278384 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:57:16.278384 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:57:16.278384 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:57:16.278384 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:57:16.278384 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:57:16.278384 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:57:16.278384 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:57:16.278384 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:57:16.278384 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:57:16.278384 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:57:16.278384 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:57:16.279298 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:57:16.279298 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:57:16.279298 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:57:16.279298 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:57:16.279298 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:57:16.279298 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:57:16.279298 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:57:16.279298 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:57:16.279298 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:57:16.279298 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:57:16.279298 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:57:16.279298 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application with Lock( Aug 24 05:57:16.279298 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:57:16.279298 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:57:16.279298 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:57:16.279298 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:57:16.280095 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:57:16.280095 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:57:16.280095 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:57:16.280095 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:57:16.280095 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:57:16.280095 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:57:16.280095 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:57:16.280095 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:57:16.280095 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:57:16.280095 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:57:16.280095 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 05:57:16.280095 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Aug 24 05:57:16.280095 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 170/342] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Aug 24 05:57:15 2022] GET /identity/v3/roles/reader => generated 84 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:57:16.287828 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59589129-2e4d-410e-89cf-8885e8ac686a None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:16.356878 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:16.380853 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5265295-afd9-4b77-8882-3e4d9d8b1c1d None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:16.409435 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59589129-2e4d-410e-89cf-8885e8ac686a None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:16.433499 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27610dc-0e1b-47e9-9d60-755f28c5dc72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:16.434114 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27610dc-0e1b-47e9-9d60-755f28c5dc72 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:16.434416 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27610dc-0e1b-47e9-9d60-755f28c5dc72 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:16.435904 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27610dc-0e1b-47e9-9d60-755f28c5dc72 None admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:16.437904 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27610dc-0e1b-47e9-9d60-755f28c5dc72 None admin] RBAC: Adding query filter params (name=member) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:57:16.438222 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27610dc-0e1b-47e9-9d60-755f28c5dc72 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:16.440156 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27610dc-0e1b-47e9-9d60-755f28c5dc72 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:16.443553 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:16.455141 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 173/343] 10.222.0.62 () {56 vars in 1101 bytes} [Wed Aug 24 05:57:16 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:57:16.455833 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27610dc-0e1b-47e9-9d60-755f28c5dc72 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:16.469588 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e946c7cc-fad1-4ac9-9156-26a62c790a49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:16.469588 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e946c7cc-fad1-4ac9-9156-26a62c790a49 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:16.469588 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e946c7cc-fad1-4ac9-9156-26a62c790a49 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:16.469588 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e946c7cc-fad1-4ac9-9156-26a62c790a49 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:16.473203 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e946c7cc-fad1-4ac9-9156-26a62c790a49 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:57:16.478732 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application [None req-e946c7cc-fad1-4ac9-9156-26a62c790a49 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 05:57:16.478732 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:57:16.478732 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:57:16.478732 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:57:16.478732 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:57:16.478732 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:57:16.478732 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:57:16.478732 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:57:16.478732 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:57:16.478732 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:57:16.478732 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:57:16.478732 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:57:16.478732 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:57:16.478732 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:57:16.478732 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:57:16.478732 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:57:16.479608 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:57:16.479608 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:57:16.479608 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:57:16.479608 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:57:16.479608 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:57:16.479608 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:57:16.479608 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:57:16.479608 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:57:16.479608 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:57:16.479608 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:57:16.479608 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:57:16.479608 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application with Lock( Aug 24 05:57:16.479608 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:57:16.479608 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:57:16.479608 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:57:16.479608 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:57:16.480421 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:57:16.480421 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:57:16.480421 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:57:16.480421 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:57:16.480421 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:57:16.480421 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:57:16.480421 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:57:16.480421 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:57:16.480421 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:57:16.480421 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:57:16.480421 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 05:57:16.480421 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Aug 24 05:57:16.482832 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 171/344] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Aug 24 05:57:16 2022] GET /identity/v3/roles/member => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 05:57:16.484591 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb9f871-9cc1-4f5e-a88a-aff610ccf403 None None] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:16.485076 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb9f871-9cc1-4f5e-a88a-aff610ccf403 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:16.485601 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb9f871-9cc1-4f5e-a88a-aff610ccf403 None None] PATH_INFO: `/` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:16.487234 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 172/345] 10.222.0.62 () {54 vars in 927 bytes} [Wed Aug 24 05:57:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:57:16.492481 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb9f871-9cc1-4f5e-a88a-aff610ccf403 None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:16.581224 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27610dc-0e1b-47e9-9d60-755f28c5dc72 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:16.616957 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb9f871-9cc1-4f5e-a88a-aff610ccf403 None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:16.636643 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db39100c-7184-4179-a128-e1bc2cafde03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:16.637258 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db39100c-7184-4179-a128-e1bc2cafde03 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:16.637536 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db39100c-7184-4179-a128-e1bc2cafde03 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:16.637805 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db39100c-7184-4179-a128-e1bc2cafde03 None admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:16.638467 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db39100c-7184-4179-a128-e1bc2cafde03 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:57:16.638767 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db39100c-7184-4179-a128-e1bc2cafde03 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:16.643536 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db39100c-7184-4179-a128-e1bc2cafde03 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:16.646159 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:16.655409 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 174/346] 10.222.0.62 () {56 vars in 1101 bytes} [Wed Aug 24 05:57:16 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:57:16.656281 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db39100c-7184-4179-a128-e1bc2cafde03 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:16.676386 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecb5f9b-7be9-45d7-be8a-d061b52bccf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:16.677930 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecb5f9b-7be9-45d7-be8a-d061b52bccf2 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:16.678415 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecb5f9b-7be9-45d7-be8a-d061b52bccf2 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:16.679176 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecb5f9b-7be9-45d7-be8a-d061b52bccf2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:16.734790 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:16.761202 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ecb5f9b-7be9-45d7-be8a-d061b52bccf2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:16.762567 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ecb5f9b-7be9-45d7-be8a-d061b52bccf2 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:16.783938 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db39100c-7184-4179-a128-e1bc2cafde03 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:16.827683 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386c42bf-6295-4df8-bcdb-54268f78dba6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:16.828932 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386c42bf-6295-4df8-bcdb-54268f78dba6 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:16.829654 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386c42bf-6295-4df8-bcdb-54268f78dba6 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:16.830134 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386c42bf-6295-4df8-bcdb-54268f78dba6 None admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:16.831558 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386c42bf-6295-4df8-bcdb-54268f78dba6 None admin] RBAC: Adding query filter params (name=member) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:57:16.832029 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386c42bf-6295-4df8-bcdb-54268f78dba6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:16.834250 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386c42bf-6295-4df8-bcdb-54268f78dba6 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:16.845553 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 175/347] 10.222.0.62 () {56 vars in 1101 bytes} [Wed Aug 24 05:57:16 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:57:16.847218 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ddda1e-4d3e-408a-86fe-cc3d3f094841 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:16.847650 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ddda1e-4d3e-408a-86fe-cc3d3f094841 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:16.852525 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ddda1e-4d3e-408a-86fe-cc3d3f094841 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:16.880642 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-b1ddda1e-4d3e-408a-86fe-cc3d3f094841 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:57:16.882768 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-b1ddda1e-4d3e-408a-86fe-cc3d3f094841 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:57:16.894464 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 173/348] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Aug 24 05:57:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 05:57:16.895438 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecb5f9b-7be9-45d7-be8a-d061b52bccf2 None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:16.920895 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:16.972061 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 176/349] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Aug 24 05:57:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 05:57:16.972061 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ddda1e-4d3e-408a-86fe-cc3d3f094841 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:17.049533 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecb5f9b-7be9-45d7-be8a-d061b52bccf2 None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:17.057708 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:17.090599 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58c5a18-0119-428e-b180-837e84a47922 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:17.091182 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58c5a18-0119-428e-b180-837e84a47922 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:17.091512 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58c5a18-0119-428e-b180-837e84a47922 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:17.091790 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58c5a18-0119-428e-b180-837e84a47922 None admin] PATH_INFO: `/v3/users/5a808a34e0d24205a4b92f987a85eacf` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:17.092947 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58c5a18-0119-428e-b180-837e84a47922 None admin] RBAC: Authorizing `identity:get_user(user_id=5a808a34e0d24205a4b92f987a85eacf)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:17.094378 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58c5a18-0119-428e-b180-837e84a47922 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:17.096357 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 174/350] 10.222.0.62 () {56 vars in 1177 bytes} [Wed Aug 24 05:57:16 2022] GET /identity/v3/users/5a808a34e0d24205a4b92f987a85eacf => generated 293 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:57:17.097168 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58c5a18-0119-428e-b180-837e84a47922 None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:17.113339 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ddda1e-4d3e-408a-86fe-cc3d3f094841 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:17.149676 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482aa2cc-709e-4230-8bbc-1afab3343d21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:17.150290 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482aa2cc-709e-4230-8bbc-1afab3343d21 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:17.150563 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482aa2cc-709e-4230-8bbc-1afab3343d21 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:17.150830 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482aa2cc-709e-4230-8bbc-1afab3343d21 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:17.193700 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:17.214639 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58c5a18-0119-428e-b180-837e84a47922 None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:17.252644 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482aa2cc-709e-4230-8bbc-1afab3343d21 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:17.253214 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a17e2d-a171-435c-a085-8bbe234497fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:17.253845 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a17e2d-a171-435c-a085-8bbe234497fe None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:17.254124 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a17e2d-a171-435c-a085-8bbe234497fe None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:17.254402 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a17e2d-a171-435c-a085-8bbe234497fe None admin] PATH_INFO: `/v3/projects/6d7fd5ca249a4df5aba301beb23e2eb7` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:17.255335 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482aa2cc-709e-4230-8bbc-1afab3343d21 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:17.255637 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a17e2d-a171-435c-a085-8bbe234497fe None admin] RBAC: Authorizing `identity:get_project(project_id=6d7fd5ca249a4df5aba301beb23e2eb7)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:17.257164 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a17e2d-a171-435c-a085-8bbe234497fe None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:17.260116 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 175/351] 10.222.0.62 () {56 vars in 1186 bytes} [Wed Aug 24 05:57:17 2022] GET /identity/v3/projects/6d7fd5ca249a4df5aba301beb23e2eb7 => generated 298 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:57:17.261305 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3b1e8a-9363-4082-9222-bde0f5a5daf4 None None] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:17.261586 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3b1e8a-9363-4082-9222-bde0f5a5daf4 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:17.261863 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3b1e8a-9363-4082-9222-bde0f5a5daf4 None None] PATH_INFO: `/` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:17.263307 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 176/352] 10.222.0.62 () {54 vars in 927 bytes} [Wed Aug 24 05:57:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:57:17.264004 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3b1e8a-9363-4082-9222-bde0f5a5daf4 None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:17.333520 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:17.397590 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3b1e8a-9363-4082-9222-bde0f5a5daf4 None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:17.398078 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 177/353] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Aug 24 05:57:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 05:57:17.398816 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482aa2cc-709e-4230-8bbc-1afab3343d21 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:17.435159 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b347f3e6-1158-40b4-baef-77cd1cab6c68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:17.435767 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b347f3e6-1158-40b4-baef-77cd1cab6c68 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:17.436046 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b347f3e6-1158-40b4-baef-77cd1cab6c68 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:17.436317 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b347f3e6-1158-40b4-baef-77cd1cab6c68 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:17.437102 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b347f3e6-1158-40b4-baef-77cd1cab6c68 None admin] RBAC: Adding query filter params (role.id=e2f40474c6de4f5fadfa7efbe4087971, user.id=5a808a34e0d24205a4b92f987a85eacf, scope.system=all) {{(pid=81851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:57:17.437409 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b347f3e6-1158-40b4-baef-77cd1cab6c68 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:17.438892 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b347f3e6-1158-40b4-baef-77cd1cab6c68 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:17.452645 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 177/354] 10.222.0.62 () {56 vars in 1308 bytes} [Wed Aug 24 05:57:17 2022] GET /identity/v3/role_assignments?user.id=5a808a34e0d24205a4b92f987a85eacf&scope.system=all&role.id=e2f40474c6de4f5fadfa7efbe4087971 => generated 510 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:57:17.453706 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88494614-4995-412a-8f8b-bddb477117f1 None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:17.453986 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88494614-4995-412a-8f8b-bddb477117f1 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:17.454296 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88494614-4995-412a-8f8b-bddb477117f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:17.477336 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-88494614-4995-412a-8f8b-bddb477117f1 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:57:17.479554 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-88494614-4995-412a-8f8b-bddb477117f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:57:17.481938 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:17.523158 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482aa2cc-709e-4230-8bbc-1afab3343d21 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:17.573318 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c01f98-06e9-476b-8bf8-523b0a46d909 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:17.574213 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c01f98-06e9-476b-8bf8-523b0a46d909 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:17.574742 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c01f98-06e9-476b-8bf8-523b0a46d909 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:17.575148 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 178/355] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Aug 24 05:57:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 05:57:17.575312 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c01f98-06e9-476b-8bf8-523b0a46d909 None admin] PATH_INFO: `/v3/groups/cec1f56499194c9fb4ba718917f95478` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:17.576433 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88494614-4995-412a-8f8b-bddb477117f1 None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:17.589357 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c01f98-06e9-476b-8bf8-523b0a46d909 None admin] RBAC: Authorizing `identity:get_group(group_id=cec1f56499194c9fb4ba718917f95478)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:17.591098 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c01f98-06e9-476b-8bf8-523b0a46d909 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:17.593026 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 178/356] 10.222.0.62 () {56 vars in 1180 bytes} [Wed Aug 24 05:57:17 2022] GET /identity/v3/groups/cec1f56499194c9fb4ba718917f95478 => generated 226 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:57:17.593998 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c01f98-06e9-476b-8bf8-523b0a46d909 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:17.623623 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:17.717983 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88494614-4995-412a-8f8b-bddb477117f1 None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:17.745213 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c01f98-06e9-476b-8bf8-523b0a46d909 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:17.766675 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7726ab79-b493-4104-a967-aa4ace671bf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:17.767369 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7726ab79-b493-4104-a967-aa4ace671bf0 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:17.767656 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7726ab79-b493-4104-a967-aa4ace671bf0 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:17.767935 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7726ab79-b493-4104-a967-aa4ace671bf0 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:17.779257 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:17.784184 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7726ab79-b493-4104-a967-aa4ace671bf0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:57:17.789372 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application [None req-7726ab79-b493-4104-a967-aa4ace671bf0 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 05:57:17.789372 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:57:17.789372 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:57:17.789372 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:57:17.789372 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:57:17.789372 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:57:17.789372 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:57:17.789372 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:57:17.789372 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:57:17.789372 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:57:17.789372 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:57:17.789372 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:57:17.789372 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:57:17.789372 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:57:17.789372 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:57:17.789372 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:57:17.790251 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:57:17.790251 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:57:17.790251 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:57:17.790251 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:57:17.790251 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:57:17.790251 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:57:17.790251 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:57:17.790251 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:57:17.790251 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:57:17.790251 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:57:17.790251 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:57:17.790251 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application with Lock( Aug 24 05:57:17.790251 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:57:17.790251 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:57:17.790251 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:57:17.790251 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:57:17.791042 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:57:17.791042 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:57:17.791042 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:57:17.791042 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:57:17.791042 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:57:17.791042 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:57:17.791042 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:57:17.791042 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:57:17.791042 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:57:17.791042 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:57:17.791042 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 05:57:17.791042 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Aug 24 05:57:17.791042 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 179/357] 10.222.0.62 () {56 vars in 1114 bytes} [Wed Aug 24 05:57:17 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:57:17.791704 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7726ab79-b493-4104-a967-aa4ace671bf0 None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:17.799062 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84aacd6-95cb-422a-ac5a-4f1ef7292cfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:17.802405 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84aacd6-95cb-422a-ac5a-4f1ef7292cfc None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:17.802405 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84aacd6-95cb-422a-ac5a-4f1ef7292cfc None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:17.802405 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84aacd6-95cb-422a-ac5a-4f1ef7292cfc None admin] PATH_INFO: `/v3/projects/9391bbe626b44799942ef25c41bc33ad` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:17.802405 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84aacd6-95cb-422a-ac5a-4f1ef7292cfc None admin] RBAC: Authorizing `identity:get_project(project_id=9391bbe626b44799942ef25c41bc33ad)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:17.803252 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84aacd6-95cb-422a-ac5a-4f1ef7292cfc None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:17.806005 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 179/358] 10.222.0.62 () {56 vars in 1186 bytes} [Wed Aug 24 05:57:17 2022] GET /identity/v3/projects/9391bbe626b44799942ef25c41bc33ad => generated 302 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:57:17.807326 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8215d3-e257-4bc0-8b5e-24d660aa917e None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:17.807673 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8215d3-e257-4bc0-8b5e-24d660aa917e None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:17.808015 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8215d3-e257-4bc0-8b5e-24d660aa917e None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:17.809303 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 180/359] 10.222.0.62 () {54 vars in 927 bytes} [Wed Aug 24 05:57:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:57:17.810160 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8215d3-e257-4bc0-8b5e-24d660aa917e None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:17.906972 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:17.920938 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7726ab79-b493-4104-a967-aa4ace671bf0 None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:17.927583 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8215d3-e257-4bc0-8b5e-24d660aa917e None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:17.960942 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffacce5-e6b2-4e1f-8bc7-e591fe888fa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:17.961600 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffacce5-e6b2-4e1f-8bc7-e591fe888fa6 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:17.961889 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffacce5-e6b2-4e1f-8bc7-e591fe888fa6 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:17.962165 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffacce5-e6b2-4e1f-8bc7-e591fe888fa6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:17.962895 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffacce5-e6b2-4e1f-8bc7-e591fe888fa6 None admin] RBAC: Adding query filter params (group.id=cec1f56499194c9fb4ba718917f95478, role.id=580aa7dbb38d4cff8f9d1bb8983f99ef, scope.project.id=6d7fd5ca249a4df5aba301beb23e2eb7) {{(pid=81851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:57:17.963205 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffacce5-e6b2-4e1f-8bc7-e591fe888fa6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:17.964680 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffacce5-e6b2-4e1f-8bc7-e591fe888fa6 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:17.966014 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d139878b-9288-486d-a7f7-08fce3385c62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:17.966617 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d139878b-9288-486d-a7f7-08fce3385c62 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:17.966890 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d139878b-9288-486d-a7f7-08fce3385c62 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:17.968564 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d139878b-9288-486d-a7f7-08fce3385c62 None admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:17.968564 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d139878b-9288-486d-a7f7-08fce3385c62 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:57:17.968564 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d139878b-9288-486d-a7f7-08fce3385c62 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:17.969520 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d139878b-9288-486d-a7f7-08fce3385c62 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:17.975065 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 180/360] 10.222.0.62 () {56 vars in 1376 bytes} [Wed Aug 24 05:57:17 2022] GET /identity/v3/role_assignments?group.id=cec1f56499194c9fb4ba718917f95478&scope.project.id=6d7fd5ca249a4df5aba301beb23e2eb7&role.id=580aa7dbb38d4cff8f9d1bb8983f99ef => generated 260 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:57:17.977586 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add82890-1997-48cd-a247-11ca5163a051 None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:17.977813 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add82890-1997-48cd-a247-11ca5163a051 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:17.978070 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add82890-1997-48cd-a247-11ca5163a051 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:17.983860 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 181/361] 10.222.0.62 () {56 vars in 1111 bytes} [Wed Aug 24 05:57:17 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:57:17.984814 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d139878b-9288-486d-a7f7-08fce3385c62 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:18.002195 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-add82890-1997-48cd-a247-11ca5163a051 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:57:18.004560 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-add82890-1997-48cd-a247-11ca5163a051 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:57:18.085450 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:18.101671 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 181/362] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Aug 24 05:57:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 05:57:18.102353 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add82890-1997-48cd-a247-11ca5163a051 None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:18.122629 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d139878b-9288-486d-a7f7-08fce3385c62 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:18.171776 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7f1d6e-d3b9-487b-b724-9a8b4bbf6f09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:18.172343 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7f1d6e-d3b9-487b-b724-9a8b4bbf6f09 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:18.172658 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7f1d6e-d3b9-487b-b724-9a8b4bbf6f09 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:18.172932 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7f1d6e-d3b9-487b-b724-9a8b4bbf6f09 None admin] PATH_INFO: `/v3/groups/cec1f56499194c9fb4ba718917f95478` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:18.174944 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7f1d6e-d3b9-487b-b724-9a8b4bbf6f09 None admin] RBAC: Authorizing `identity:get_group(group_id=cec1f56499194c9fb4ba718917f95478)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:18.177144 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7f1d6e-d3b9-487b-b724-9a8b4bbf6f09 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:18.180273 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 182/363] 10.222.0.62 () {56 vars in 1180 bytes} [Wed Aug 24 05:57:17 2022] GET /identity/v3/groups/cec1f56499194c9fb4ba718917f95478 => generated 226 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:57:18.181853 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7f1d6e-d3b9-487b-b724-9a8b4bbf6f09 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:18.220471 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:18.226237 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add82890-1997-48cd-a247-11ca5163a051 None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:18.265854 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40723043-9807-4d39-8811-3d85942faa65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:18.266627 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40723043-9807-4d39-8811-3d85942faa65 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:18.266941 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40723043-9807-4d39-8811-3d85942faa65 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:18.267208 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40723043-9807-4d39-8811-3d85942faa65 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:18.278993 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40723043-9807-4d39-8811-3d85942faa65 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:57:18.284582 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application [None req-40723043-9807-4d39-8811-3d85942faa65 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 05:57:18.284582 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:57:18.284582 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:57:18.284582 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:57:18.284582 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:57:18.284582 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:57:18.284582 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:57:18.284582 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:57:18.284582 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:57:18.284582 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:57:18.284582 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:57:18.284582 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:57:18.284582 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:57:18.284582 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:57:18.284582 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:57:18.284582 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:57:18.285666 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:57:18.285666 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:57:18.285666 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:57:18.285666 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:57:18.285666 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:57:18.285666 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:57:18.285666 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:57:18.285666 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:57:18.285666 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:57:18.285666 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:57:18.285666 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:57:18.285666 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application with Lock( Aug 24 05:57:18.285666 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:57:18.285666 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:57:18.285666 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:57:18.285666 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:57:18.286654 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:57:18.286654 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:57:18.286654 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:57:18.286654 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:57:18.286654 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:57:18.286654 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:57:18.286654 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:57:18.286654 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:57:18.286654 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:57:18.286654 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:57:18.286654 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 05:57:18.286654 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Aug 24 05:57:18.286654 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 182/364] 10.222.0.62 () {56 vars in 1114 bytes} [Wed Aug 24 05:57:18 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:57:18.287443 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40723043-9807-4d39-8811-3d85942faa65 None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:18.326708 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7f1d6e-d3b9-487b-b724-9a8b4bbf6f09 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:18.358864 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:18.376968 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394cbf6b-b137-4cf4-9fca-bad0b658ed7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:18.378371 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394cbf6b-b137-4cf4-9fca-bad0b658ed7b None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:18.378658 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394cbf6b-b137-4cf4-9fca-bad0b658ed7b None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:18.378935 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394cbf6b-b137-4cf4-9fca-bad0b658ed7b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:18.414916 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40723043-9807-4d39-8811-3d85942faa65 None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:18.465232 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d193d57-8d98-4c00-b0af-c16fa6f6f446 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:18.465844 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d193d57-8d98-4c00-b0af-c16fa6f6f446 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:18.466623 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d193d57-8d98-4c00-b0af-c16fa6f6f446 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:18.466900 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d193d57-8d98-4c00-b0af-c16fa6f6f446 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:18.467627 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d193d57-8d98-4c00-b0af-c16fa6f6f446 None admin] RBAC: Adding query filter params (group.id=cec1f56499194c9fb4ba718917f95478, role.id=580aa7dbb38d4cff8f9d1bb8983f99ef, scope.project.id=9391bbe626b44799942ef25c41bc33ad) {{(pid=81851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:57:18.468549 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d193d57-8d98-4c00-b0af-c16fa6f6f446 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:18.470935 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d193d57-8d98-4c00-b0af-c16fa6f6f446 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:18.471888 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394cbf6b-b137-4cf4-9fca-bad0b658ed7b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:18.473414 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394cbf6b-b137-4cf4-9fca-bad0b658ed7b None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:18.477273 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 183/365] 10.222.0.62 () {56 vars in 1376 bytes} [Wed Aug 24 05:57:18 2022] GET /identity/v3/role_assignments?group.id=cec1f56499194c9fb4ba718917f95478&scope.project.id=9391bbe626b44799942ef25c41bc33ad&role.id=580aa7dbb38d4cff8f9d1bb8983f99ef => generated 260 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:57:18.477901 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d193d57-8d98-4c00-b0af-c16fa6f6f446 None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:18.491947 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:18.614085 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 183/366] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Aug 24 05:57:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 05:57:18.614991 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394cbf6b-b137-4cf4-9fca-bad0b658ed7b None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:18.628726 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d193d57-8d98-4c00-b0af-c16fa6f6f446 None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:18.649457 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:18.669608 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf67d5e-b6a2-44b9-a327-fcec2542f36a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:18.670477 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf67d5e-b6a2-44b9-a327-fcec2542f36a None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:18.670755 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf67d5e-b6a2-44b9-a327-fcec2542f36a None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:18.671025 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf67d5e-b6a2-44b9-a327-fcec2542f36a None admin] PATH_INFO: `/v3/roles` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:18.671681 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf67d5e-b6a2-44b9-a327-fcec2542f36a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:57:18.672085 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf67d5e-b6a2-44b9-a327-fcec2542f36a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:18.673585 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf67d5e-b6a2-44b9-a327-fcec2542f36a None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:18.683528 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 184/367] 10.222.0.62 () {56 vars in 1111 bytes} [Wed Aug 24 05:57:18 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:57:18.689232 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf67d5e-b6a2-44b9-a327-fcec2542f36a None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:18.724607 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394cbf6b-b137-4cf4-9fca-bad0b658ed7b None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:18.739353 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:18.761554 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9e819d-f6b0-45dc-b10d-65a60d5ad2b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:18.762189 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9e819d-f6b0-45dc-b10d-65a60d5ad2b1 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:18.762464 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9e819d-f6b0-45dc-b10d-65a60d5ad2b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:18.762821 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9e819d-f6b0-45dc-b10d-65a60d5ad2b1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:18.776065 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9e819d-f6b0-45dc-b10d-65a60d5ad2b1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:57:18.781605 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-6f9e819d-f6b0-45dc-b10d-65a60d5ad2b1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 05:57:18.781605 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:57:18.781605 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:57:18.781605 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:57:18.781605 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:57:18.781605 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:57:18.781605 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:57:18.781605 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:57:18.781605 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:57:18.781605 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:57:18.781605 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:57:18.781605 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:57:18.781605 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:57:18.781605 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:57:18.781605 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:57:18.781605 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:57:18.782706 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:57:18.782706 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:57:18.782706 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:57:18.782706 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:57:18.782706 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:57:18.782706 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:57:18.782706 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:57:18.782706 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:57:18.782706 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:57:18.782706 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:57:18.782706 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:57:18.782706 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 24 05:57:18.782706 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:57:18.782706 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:57:18.782706 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:57:18.782706 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:57:18.783551 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:57:18.783551 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:57:18.783551 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:57:18.783551 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:57:18.783551 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:57:18.783551 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:57:18.783551 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:57:18.783551 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:57:18.783551 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:57:18.783551 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:57:18.783551 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 05:57:18.783551 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 24 05:57:18.783551 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 184/368] 10.222.0.62 () {56 vars in 1096 bytes} [Wed Aug 24 05:57:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:57:18.784413 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9e819d-f6b0-45dc-b10d-65a60d5ad2b1 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:18.812062 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf67d5e-b6a2-44b9-a327-fcec2542f36a None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:18.854273 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b9b403-4f2b-4957-82f2-8315a449a6d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:18.855212 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b9b403-4f2b-4957-82f2-8315a449a6d3 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:18.855717 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b9b403-4f2b-4957-82f2-8315a449a6d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:18.856630 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b9b403-4f2b-4957-82f2-8315a449a6d3 None admin] PATH_INFO: `/v3/projects/6d7fd5ca249a4df5aba301beb23e2eb7` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:18.858135 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b9b403-4f2b-4957-82f2-8315a449a6d3 None admin] RBAC: Authorizing `identity:get_project(project_id=6d7fd5ca249a4df5aba301beb23e2eb7)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:18.859820 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b9b403-4f2b-4957-82f2-8315a449a6d3 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:18.862000 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 185/369] 10.222.0.62 () {56 vars in 1186 bytes} [Wed Aug 24 05:57:18 2022] GET /identity/v3/projects/6d7fd5ca249a4df5aba301beb23e2eb7 => generated 298 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:57:18.863039 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b9b403-4f2b-4957-82f2-8315a449a6d3 None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:18.918988 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9e819d-f6b0-45dc-b10d-65a60d5ad2b1 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:18.923697 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:18.965021 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe0f592-b4e0-466d-8458-96c20f55138e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:18.966020 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe0f592-b4e0-466d-8458-96c20f55138e None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:18.966501 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe0f592-b4e0-466d-8458-96c20f55138e None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:18.966994 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe0f592-b4e0-466d-8458-96c20f55138e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:18.998453 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b9b403-4f2b-4957-82f2-8315a449a6d3 None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:19.018463 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:19.044336 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6628b895-c44e-4e5e-bdbe-7b6336f8c0e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:19.044738 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6628b895-c44e-4e5e-bdbe-7b6336f8c0e1 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:19.045021 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6628b895-c44e-4e5e-bdbe-7b6336f8c0e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:19.045682 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6628b895-c44e-4e5e-bdbe-7b6336f8c0e1 None admin] PATH_INFO: `/v3/roles` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:19.046395 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6628b895-c44e-4e5e-bdbe-7b6336f8c0e1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:57:19.046696 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6628b895-c44e-4e5e-bdbe-7b6336f8c0e1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:19.047011 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe0f592-b4e0-466d-8458-96c20f55138e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:19.048607 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6628b895-c44e-4e5e-bdbe-7b6336f8c0e1 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:19.049473 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe0f592-b4e0-466d-8458-96c20f55138e None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:19.060007 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 186/370] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Aug 24 05:57:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:57:19.060860 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6628b895-c44e-4e5e-bdbe-7b6336f8c0e1 None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:19.164071 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 185/371] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Aug 24 05:57:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 05:57:19.165389 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe0f592-b4e0-466d-8458-96c20f55138e None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:19.178910 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6628b895-c44e-4e5e-bdbe-7b6336f8c0e1 None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:19.189372 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:19.217281 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4894986-a68b-4f0e-a0bc-b01bf817e8d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:19.218417 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4894986-a68b-4f0e-a0bc-b01bf817e8d3 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:19.219081 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4894986-a68b-4f0e-a0bc-b01bf817e8d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:19.219558 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4894986-a68b-4f0e-a0bc-b01bf817e8d3 None admin] PATH_INFO: `/v3/groups/cec1f56499194c9fb4ba718917f95478` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:19.221384 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4894986-a68b-4f0e-a0bc-b01bf817e8d3 None admin] RBAC: Authorizing `identity:get_group(group_id=cec1f56499194c9fb4ba718917f95478)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:19.223833 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4894986-a68b-4f0e-a0bc-b01bf817e8d3 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:19.226473 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 187/372] 10.222.0.62 () {56 vars in 1180 bytes} [Wed Aug 24 05:57:19 2022] GET /identity/v3/groups/cec1f56499194c9fb4ba718917f95478 => generated 226 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:57:19.227654 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4894986-a68b-4f0e-a0bc-b01bf817e8d3 None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:19.254270 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:19.279669 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe0f592-b4e0-466d-8458-96c20f55138e None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:19.316662 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12dc5c7-0ace-4892-ab64-cf374fb6a821 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:19.317465 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12dc5c7-0ace-4892-ab64-cf374fb6a821 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:19.317857 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12dc5c7-0ace-4892-ab64-cf374fb6a821 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:19.318227 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12dc5c7-0ace-4892-ab64-cf374fb6a821 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:19.347243 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4894986-a68b-4f0e-a0bc-b01bf817e8d3 None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:19.384703 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00854d0c-33ea-480e-bd39-41c36299c3ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:19.385654 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00854d0c-33ea-480e-bd39-41c36299c3ce None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:19.386151 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00854d0c-33ea-480e-bd39-41c36299c3ce None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:19.386660 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00854d0c-33ea-480e-bd39-41c36299c3ce None admin] PATH_INFO: `/v3/projects/9391bbe626b44799942ef25c41bc33ad` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:19.388066 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00854d0c-33ea-480e-bd39-41c36299c3ce None admin] RBAC: Authorizing `identity:get_project(project_id=9391bbe626b44799942ef25c41bc33ad)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:19.389633 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12dc5c7-0ace-4892-ab64-cf374fb6a821 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:19.389993 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00854d0c-33ea-480e-bd39-41c36299c3ce None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:19.391100 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12dc5c7-0ace-4892-ab64-cf374fb6a821 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:19.392194 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 188/373] 10.222.0.62 () {56 vars in 1186 bytes} [Wed Aug 24 05:57:19 2022] GET /identity/v3/projects/9391bbe626b44799942ef25c41bc33ad => generated 302 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:57:19.393492 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00854d0c-33ea-480e-bd39-41c36299c3ce None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:19.448404 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:19.468467 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:19.497393 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 186/374] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Aug 24 05:57:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 05:57:19.498236 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12dc5c7-0ace-4892-ab64-cf374fb6a821 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:19.511945 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00854d0c-33ea-480e-bd39-41c36299c3ce None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:19.554043 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddb1480-4929-4117-b1c8-0c4d2ccf74ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:19.555091 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddb1480-4929-4117-b1c8-0c4d2ccf74ac None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:19.555658 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddb1480-4929-4117-b1c8-0c4d2ccf74ac None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:19.556118 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddb1480-4929-4117-b1c8-0c4d2ccf74ac None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:19.557113 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddb1480-4929-4117-b1c8-0c4d2ccf74ac None admin] RBAC: Adding query filter params (group.id=cec1f56499194c9fb4ba718917f95478, role.id=ba3c2c3cb98c424f9ece19903a59c53c, scope.project.id=6d7fd5ca249a4df5aba301beb23e2eb7) {{(pid=81851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:57:19.557617 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddb1480-4929-4117-b1c8-0c4d2ccf74ac None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:19.559336 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddb1480-4929-4117-b1c8-0c4d2ccf74ac None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:19.572674 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 189/375] 10.222.0.62 () {56 vars in 1376 bytes} [Wed Aug 24 05:57:19 2022] GET /identity/v3/role_assignments?group.id=cec1f56499194c9fb4ba718917f95478&scope.project.id=6d7fd5ca249a4df5aba301beb23e2eb7&role.id=ba3c2c3cb98c424f9ece19903a59c53c => generated 260 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:57:19.574574 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddb1480-4929-4117-b1c8-0c4d2ccf74ac None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:19.603608 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12dc5c7-0ace-4892-ab64-cf374fb6a821 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:19.670099 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cceac45-efc1-49c5-85f4-fedacc0e430f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:19.671005 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cceac45-efc1-49c5-85f4-fedacc0e430f None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:19.671282 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cceac45-efc1-49c5-85f4-fedacc0e430f None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:19.671549 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cceac45-efc1-49c5-85f4-fedacc0e430f None admin] PATH_INFO: `/v3/groups/cec1f56499194c9fb4ba718917f95478` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:19.672890 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cceac45-efc1-49c5-85f4-fedacc0e430f None admin] RBAC: Authorizing `identity:get_group(group_id=cec1f56499194c9fb4ba718917f95478)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:19.676393 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cceac45-efc1-49c5-85f4-fedacc0e430f None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:19.680117 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 187/376] 10.222.0.62 () {56 vars in 1180 bytes} [Wed Aug 24 05:57:19 2022] GET /identity/v3/groups/cec1f56499194c9fb4ba718917f95478 => generated 226 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:57:19.686666 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cceac45-efc1-49c5-85f4-fedacc0e430f None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:19.755576 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddb1480-4929-4117-b1c8-0c4d2ccf74ac None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:19.779467 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:19.782956 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:19.853288 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99a59be-ea02-4830-82e7-3f04b3e62e37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:19.853944 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99a59be-ea02-4830-82e7-3f04b3e62e37 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:19.854223 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99a59be-ea02-4830-82e7-3f04b3e62e37 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:19.854491 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99a59be-ea02-4830-82e7-3f04b3e62e37 None admin] PATH_INFO: `/v3/projects/632e93ce840f4853b64b4d659536e045` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:19.855683 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99a59be-ea02-4830-82e7-3f04b3e62e37 None admin] RBAC: Authorizing `identity:get_project(project_id=632e93ce840f4853b64b4d659536e045)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:19.857971 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99a59be-ea02-4830-82e7-3f04b3e62e37 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:19.859016 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 190/377] 10.222.0.62 () {56 vars in 1186 bytes} [Wed Aug 24 05:57:19 2022] GET /identity/v3/projects/632e93ce840f4853b64b4d659536e045 => generated 344 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:57:19.861570 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cceac45-efc1-49c5-85f4-fedacc0e430f None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:19.867727 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99a59be-ea02-4830-82e7-3f04b3e62e37 None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:19.929320 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a11c077-47b4-4be4-84d1-702d27af756a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:19.931502 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a11c077-47b4-4be4-84d1-702d27af756a None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:19.931979 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a11c077-47b4-4be4-84d1-702d27af756a None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:19.933323 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a11c077-47b4-4be4-84d1-702d27af756a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:19.934288 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a11c077-47b4-4be4-84d1-702d27af756a None admin] RBAC: Adding query filter params (group.id=cec1f56499194c9fb4ba718917f95478, role.id=ba3c2c3cb98c424f9ece19903a59c53c, scope.project.id=9391bbe626b44799942ef25c41bc33ad) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:57:19.935870 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a11c077-47b4-4be4-84d1-702d27af756a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:19.939620 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a11c077-47b4-4be4-84d1-702d27af756a None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:19.951352 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 188/378] 10.222.0.62 () {56 vars in 1376 bytes} [Wed Aug 24 05:57:19 2022] GET /identity/v3/role_assignments?group.id=cec1f56499194c9fb4ba718917f95478&scope.project.id=9391bbe626b44799942ef25c41bc33ad&role.id=ba3c2c3cb98c424f9ece19903a59c53c => generated 260 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:57:20.022089 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99a59be-ea02-4830-82e7-3f04b3e62e37 None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:20.070138 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6074d58-7db3-455a-b8c7-9d67fd60e8c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:20.071116 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6074d58-7db3-455a-b8c7-9d67fd60e8c7 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:20.071589 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6074d58-7db3-455a-b8c7-9d67fd60e8c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:20.072050 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6074d58-7db3-455a-b8c7-9d67fd60e8c7 None admin] PATH_INFO: `/v3/groups/cbf8f7f1550c4404831928e6ad3a3393` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:20.085460 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6074d58-7db3-455a-b8c7-9d67fd60e8c7 None admin] RBAC: Authorizing `identity:get_group(group_id=cbf8f7f1550c4404831928e6ad3a3393)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:20.087402 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6074d58-7db3-455a-b8c7-9d67fd60e8c7 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:20.089431 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 191/379] 10.222.0.62 () {56 vars in 1180 bytes} [Wed Aug 24 05:57:19 2022] GET /identity/v3/groups/cbf8f7f1550c4404831928e6ad3a3393 => generated 229 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:57:20.096432 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a11c077-47b4-4be4-84d1-702d27af756a None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:20.226082 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a11c077-47b4-4be4-84d1-702d27af756a None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:20.233286 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:20.269086 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2782c98-dc6c-40bc-b66e-80438554dd28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:20.269983 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2782c98-dc6c-40bc-b66e-80438554dd28 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:20.270268 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2782c98-dc6c-40bc-b66e-80438554dd28 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:20.270542 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2782c98-dc6c-40bc-b66e-80438554dd28 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:20.271273 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2782c98-dc6c-40bc-b66e-80438554dd28 None admin] RBAC: Adding query filter params (group.id=cbf8f7f1550c4404831928e6ad3a3393, role.id=49e47cd157d743f6905f1db07ec878a7, scope.project.id=632e93ce840f4853b64b4d659536e045) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:57:20.271581 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2782c98-dc6c-40bc-b66e-80438554dd28 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:20.273283 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2782c98-dc6c-40bc-b66e-80438554dd28 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:20.283623 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 189/380] 10.222.0.62 () {56 vars in 1376 bytes} [Wed Aug 24 05:57:20 2022] GET /identity/v3/role_assignments?group.id=cbf8f7f1550c4404831928e6ad3a3393&scope.project.id=632e93ce840f4853b64b4d659536e045&role.id=49e47cd157d743f6905f1db07ec878a7 => generated 260 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:57:20.296230 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c567459-7741-4a96-ab92-622fc6562368 None None] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:20.296619 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c567459-7741-4a96-ab92-622fc6562368 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:20.296810 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c567459-7741-4a96-ab92-622fc6562368 None None] PATH_INFO: `/` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:20.298264 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 192/381] 10.222.0.62 () {54 vars in 927 bytes} [Wed Aug 24 05:57:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:57:20.303797 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc027780-25b2-4900-9779-70a6bd48fdc7 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:20.304099 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc027780-25b2-4900-9779-70a6bd48fdc7 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:20.304352 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc027780-25b2-4900-9779-70a6bd48fdc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:20.327534 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-cc027780-25b2-4900-9779-70a6bd48fdc7 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:57:20.328778 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-cc027780-25b2-4900-9779-70a6bd48fdc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:57:20.407209 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 190/382] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Aug 24 05:57:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 05:57:20.706387 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c567459-7741-4a96-ab92-622fc6562368 None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:20.760913 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:20.833577 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c567459-7741-4a96-ab92-622fc6562368 None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:20.869979 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9d67d1-d1aa-4262-ba9a-72311b388068 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:20.870670 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9d67d1-d1aa-4262-ba9a-72311b388068 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:20.870948 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9d67d1-d1aa-4262-ba9a-72311b388068 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:20.871275 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9d67d1-d1aa-4262-ba9a-72311b388068 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:20.872379 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e874786-d3d1-43f7-b28c-bbeb2fb6db4c None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:20.872826 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e874786-d3d1-43f7-b28c-bbeb2fb6db4c None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:20.872983 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e874786-d3d1-43f7-b28c-bbeb2fb6db4c None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:20.874411 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 191/383] 10.222.0.62 () {54 vars in 927 bytes} [Wed Aug 24 05:57:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:57:20.880357 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc73f069-9152-4f25-99e4-3d17f21b2051 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:20.880676 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc73f069-9152-4f25-99e4-3d17f21b2051 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:20.880960 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc73f069-9152-4f25-99e4-3d17f21b2051 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:20.884011 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9d67d1-d1aa-4262-ba9a-72311b388068 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:57:20.889162 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application [None req-cf9d67d1-d1aa-4262-ba9a-72311b388068 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 05:57:20.889162 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:57:20.889162 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:57:20.889162 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:57:20.889162 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:57:20.889162 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:57:20.889162 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:57:20.889162 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:57:20.889162 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:57:20.889162 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:57:20.889162 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:57:20.889162 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:57:20.889162 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:57:20.889162 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:57:20.889162 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:57:20.889162 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:57:20.890049 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:57:20.890049 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:57:20.890049 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:57:20.890049 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:57:20.890049 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:57:20.890049 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:57:20.890049 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:57:20.890049 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:57:20.890049 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:57:20.890049 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:57:20.890049 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:57:20.890049 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application with Lock( Aug 24 05:57:20.890049 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:57:20.890049 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:57:20.890049 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:57:20.890049 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:57:20.890835 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:57:20.890835 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:57:20.890835 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:57:20.890835 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:57:20.890835 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:57:20.890835 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:57:20.890835 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:57:20.890835 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:57:20.890835 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:57:20.890835 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:57:20.890835 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 05:57:20.890835 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Aug 24 05:57:20.893149 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 193/384] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Aug 24 05:57:20 2022] GET /identity/v3/roles/member => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:57:20.900990 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9d67d1-d1aa-4262-ba9a-72311b388068 None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:20.921498 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-fc73f069-9152-4f25-99e4-3d17f21b2051 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:57:20.924282 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-fc73f069-9152-4f25-99e4-3d17f21b2051 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:57:21.047270 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 192/385] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Aug 24 05:57:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 05:57:21.064343 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:21.068237 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9d67d1-d1aa-4262-ba9a-72311b388068 None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:21.116523 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d411bf-1608-45bc-b10d-58a4ad62e4c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:21.117503 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d411bf-1608-45bc-b10d-58a4ad62e4c6 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:21.118001 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d411bf-1608-45bc-b10d-58a4ad62e4c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:21.118465 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d411bf-1608-45bc-b10d-58a4ad62e4c6 None admin] PATH_INFO: `/v3/roles` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:21.119313 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d411bf-1608-45bc-b10d-58a4ad62e4c6 None admin] RBAC: Adding query filter params (name=member) {{(pid=81851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:57:21.119807 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d411bf-1608-45bc-b10d-58a4ad62e4c6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:21.121379 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d411bf-1608-45bc-b10d-58a4ad62e4c6 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:21.135887 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 194/386] 10.222.0.62 () {56 vars in 1101 bytes} [Wed Aug 24 05:57:20 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:57:21.141169 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc73f069-9152-4f25-99e4-3d17f21b2051 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:21.192085 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:21.265651 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc73f069-9152-4f25-99e4-3d17f21b2051 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:21.301657 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb3aa6b-b9c0-44d5-8196-9a31019c2d61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:21.302224 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb3aa6b-b9c0-44d5-8196-9a31019c2d61 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:21.302606 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb3aa6b-b9c0-44d5-8196-9a31019c2d61 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:21.302886 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb3aa6b-b9c0-44d5-8196-9a31019c2d61 None admin] PATH_INFO: `/v3/groups/cec1f56499194c9fb4ba718917f95478` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:21.303992 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb3aa6b-b9c0-44d5-8196-9a31019c2d61 None admin] RBAC: Authorizing `identity:get_group(group_id=cec1f56499194c9fb4ba718917f95478)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:21.305466 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb3aa6b-b9c0-44d5-8196-9a31019c2d61 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:21.307048 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 193/387] 10.222.0.62 () {56 vars in 1180 bytes} [Wed Aug 24 05:57:21 2022] GET /identity/v3/groups/cec1f56499194c9fb4ba718917f95478 => generated 226 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:57:21.312616 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d411bf-1608-45bc-b10d-58a4ad62e4c6 None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:21.430197 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d411bf-1608-45bc-b10d-58a4ad62e4c6 None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:21.467953 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431d6b8f-ddb8-4a3f-ba4b-2935f86e26b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:21.468609 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431d6b8f-ddb8-4a3f-ba4b-2935f86e26b8 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:21.468890 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431d6b8f-ddb8-4a3f-ba4b-2935f86e26b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:21.469161 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431d6b8f-ddb8-4a3f-ba4b-2935f86e26b8 None admin] PATH_INFO: `/v3/projects/6d7fd5ca249a4df5aba301beb23e2eb7` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:21.470359 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431d6b8f-ddb8-4a3f-ba4b-2935f86e26b8 None admin] RBAC: Authorizing `identity:get_project(project_id=6d7fd5ca249a4df5aba301beb23e2eb7)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:21.471849 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431d6b8f-ddb8-4a3f-ba4b-2935f86e26b8 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:21.473521 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 195/388] 10.222.0.62 () {56 vars in 1186 bytes} [Wed Aug 24 05:57:21 2022] GET /identity/v3/projects/6d7fd5ca249a4df5aba301beb23e2eb7 => generated 298 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:57:21.479271 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb3aa6b-b9c0-44d5-8196-9a31019c2d61 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:21.507707 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431d6b8f-ddb8-4a3f-ba4b-2935f86e26b8 None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:21.562055 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:21.642248 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb3aa6b-b9c0-44d5-8196-9a31019c2d61 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:21.644660 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:21.684519 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431d6b8f-ddb8-4a3f-ba4b-2935f86e26b8 None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:21.684713 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab4cefc-ea56-4f0b-8a26-ad9ec11d737f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:21.684713 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab4cefc-ea56-4f0b-8a26-ad9ec11d737f None admin] REQUEST_METHOD: `PUT` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:21.684713 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab4cefc-ea56-4f0b-8a26-ad9ec11d737f None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:21.684713 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab4cefc-ea56-4f0b-8a26-ad9ec11d737f None admin] PATH_INFO: `/v3/projects/6d7fd5ca249a4df5aba301beb23e2eb7/groups/cec1f56499194c9fb4ba718917f95478/roles/580aa7dbb38d4cff8f9d1bb8983f99ef` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:21.685009 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab4cefc-ea56-4f0b-8a26-ad9ec11d737f None admin] RBAC: Authorizing `identity:create_grant(project_id=6d7fd5ca249a4df5aba301beb23e2eb7, group_id=cec1f56499194c9fb4ba718917f95478, role_id=580aa7dbb38d4cff8f9d1bb8983f99ef)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:21.685286 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab4cefc-ea56-4f0b-8a26-ad9ec11d737f None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:21.734496 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d726afdd-4393-4df0-bcd1-5fb6f8c6ae87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:21.735135 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d726afdd-4393-4df0-bcd1-5fb6f8c6ae87 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:21.736021 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d726afdd-4393-4df0-bcd1-5fb6f8c6ae87 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:21.736329 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d726afdd-4393-4df0-bcd1-5fb6f8c6ae87 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:21.747909 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d726afdd-4393-4df0-bcd1-5fb6f8c6ae87 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:57:21.752887 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application [None req-d726afdd-4393-4df0-bcd1-5fb6f8c6ae87 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 05:57:21.752887 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:57:21.752887 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:57:21.752887 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:57:21.752887 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:57:21.752887 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:57:21.752887 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:57:21.752887 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:57:21.752887 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:57:21.752887 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:57:21.752887 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:57:21.752887 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:57:21.752887 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:57:21.752887 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:57:21.752887 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:57:21.752887 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:57:21.753776 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:57:21.753776 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:57:21.753776 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:57:21.753776 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:57:21.753776 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:57:21.753776 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:57:21.753776 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:57:21.753776 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:57:21.753776 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:57:21.753776 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:57:21.753776 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:57:21.753776 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application with Lock( Aug 24 05:57:21.753776 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:57:21.753776 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:57:21.753776 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:57:21.753776 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:57:21.754735 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:57:21.754735 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:57:21.754735 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:57:21.754735 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:57:21.754735 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:57:21.754735 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:57:21.754735 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:57:21.754735 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:57:21.754735 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:57:21.754735 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:57:21.754735 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 05:57:21.754735 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Aug 24 05:57:21.754735 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 196/389] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Aug 24 05:57:21 2022] GET /identity/v3/roles/member => generated 84 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:57:21.759649 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d726afdd-4393-4df0-bcd1-5fb6f8c6ae87 None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:21.793266 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 194/390] 10.222.0.62 () {58 vars in 1442 bytes} [Wed Aug 24 05:57:21 2022] PUT /identity/v3/projects/6d7fd5ca249a4df5aba301beb23e2eb7/groups/cec1f56499194c9fb4ba718917f95478/roles/580aa7dbb38d4cff8f9d1bb8983f99ef => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 05:57:21.900917 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:21.910606 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d726afdd-4393-4df0-bcd1-5fb6f8c6ae87 None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:21.957103 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b9b6a3-063c-40f9-ace2-739652d9ac2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:21.957732 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b9b6a3-063c-40f9-ace2-739652d9ac2b None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:21.958012 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b9b6a3-063c-40f9-ace2-739652d9ac2b None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:21.958356 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b9b6a3-063c-40f9-ace2-739652d9ac2b None admin] PATH_INFO: `/v3/roles` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:21.959147 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b9b6a3-063c-40f9-ace2-739652d9ac2b None admin] RBAC: Adding query filter params (name=member) {{(pid=81851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:57:21.959504 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b9b6a3-063c-40f9-ace2-739652d9ac2b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:21.961020 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b9b6a3-063c-40f9-ace2-739652d9ac2b None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:21.970733 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 197/391] 10.222.0.62 () {56 vars in 1101 bytes} [Wed Aug 24 05:57:21 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:57:21.976245 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab4cefc-ea56-4f0b-8a26-ad9ec11d737f None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:22.092083 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab4cefc-ea56-4f0b-8a26-ad9ec11d737f None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:22.129098 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af72e6a0-8bf2-4662-86a0-867bf3ddae6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:22.129750 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af72e6a0-8bf2-4662-86a0-867bf3ddae6d None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:22.130089 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af72e6a0-8bf2-4662-86a0-867bf3ddae6d None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:22.130368 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af72e6a0-8bf2-4662-86a0-867bf3ddae6d None admin] PATH_INFO: `/v3/groups/cec1f56499194c9fb4ba718917f95478` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:22.131577 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af72e6a0-8bf2-4662-86a0-867bf3ddae6d None admin] RBAC: Authorizing `identity:get_group(group_id=cec1f56499194c9fb4ba718917f95478)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:22.133123 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af72e6a0-8bf2-4662-86a0-867bf3ddae6d None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:22.135065 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 195/392] 10.222.0.62 () {56 vars in 1180 bytes} [Wed Aug 24 05:57:21 2022] GET /identity/v3/groups/cec1f56499194c9fb4ba718917f95478 => generated 226 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:57:22.140287 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b9b6a3-063c-40f9-ace2-739652d9ac2b None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:22.145730 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be067b2-4a83-4d91-8ed5-94dd2a071e3a None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:22.146006 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be067b2-4a83-4d91-8ed5-94dd2a071e3a None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:22.146277 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be067b2-4a83-4d91-8ed5-94dd2a071e3a None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:22.147606 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 196/393] 10.222.0.62 () {54 vars in 927 bytes} [Wed Aug 24 05:57:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:57:22.153249 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e38dfc-3cc0-45aa-86d4-efb965efb336 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:22.153471 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e38dfc-3cc0-45aa-86d4-efb965efb336 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:22.153741 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e38dfc-3cc0-45aa-86d4-efb965efb336 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:22.180608 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-60e38dfc-3cc0-45aa-86d4-efb965efb336 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:57:22.181939 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-60e38dfc-3cc0-45aa-86d4-efb965efb336 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:57:22.209970 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:22.281122 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 197/394] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Aug 24 05:57:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 05:57:22.286244 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b9b6a3-063c-40f9-ace2-739652d9ac2b None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:22.323632 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5a3c27-b622-4e85-a59e-92c8c7daec80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:22.324683 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5a3c27-b622-4e85-a59e-92c8c7daec80 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:22.325178 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5a3c27-b622-4e85-a59e-92c8c7daec80 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:22.325660 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5a3c27-b622-4e85-a59e-92c8c7daec80 None admin] PATH_INFO: `/v3/projects/9391bbe626b44799942ef25c41bc33ad` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:22.327024 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5a3c27-b622-4e85-a59e-92c8c7daec80 None admin] RBAC: Authorizing `identity:get_project(project_id=9391bbe626b44799942ef25c41bc33ad)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:22.328699 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5a3c27-b622-4e85-a59e-92c8c7daec80 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:22.330864 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 198/395] 10.222.0.62 () {56 vars in 1186 bytes} [Wed Aug 24 05:57:22 2022] GET /identity/v3/projects/9391bbe626b44799942ef25c41bc33ad => generated 302 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:57:22.336195 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e38dfc-3cc0-45aa-86d4-efb965efb336 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:22.448908 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2faa33ae-59a7-4645-b32c-0db321430cf7 None None] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:22.449575 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2faa33ae-59a7-4645-b32c-0db321430cf7 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:22.450079 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2faa33ae-59a7-4645-b32c-0db321430cf7 None None] PATH_INFO: `/` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:22.451879 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 199/396] 10.222.0.62 () {54 vars in 927 bytes} [Wed Aug 24 05:57:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:57:22.456722 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72860763-55fe-443b-8bc1-ebbda64883e8 None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:22.457608 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72860763-55fe-443b-8bc1-ebbda64883e8 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:22.458115 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72860763-55fe-443b-8bc1-ebbda64883e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:22.477070 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e38dfc-3cc0-45aa-86d4-efb965efb336 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:22.486762 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-72860763-55fe-443b-8bc1-ebbda64883e8 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:57:22.488915 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-72860763-55fe-443b-8bc1-ebbda64883e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:57:22.507878 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:22.549209 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443ccfed-f6d4-4bb4-913e-e7c65b865146 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:22.549896 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443ccfed-f6d4-4bb4-913e-e7c65b865146 None admin] REQUEST_METHOD: `PUT` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:22.550214 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443ccfed-f6d4-4bb4-913e-e7c65b865146 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:22.550431 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443ccfed-f6d4-4bb4-913e-e7c65b865146 None admin] PATH_INFO: `/v3/projects/9391bbe626b44799942ef25c41bc33ad/groups/cec1f56499194c9fb4ba718917f95478/roles/580aa7dbb38d4cff8f9d1bb8983f99ef` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:22.552359 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443ccfed-f6d4-4bb4-913e-e7c65b865146 None admin] RBAC: Authorizing `identity:create_grant(project_id=9391bbe626b44799942ef25c41bc33ad, group_id=cec1f56499194c9fb4ba718917f95478, role_id=580aa7dbb38d4cff8f9d1bb8983f99ef)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:22.553819 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443ccfed-f6d4-4bb4-913e-e7c65b865146 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:22.572410 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 200/397] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Aug 24 05:57:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 05:57:22.629278 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 198/398] 10.222.0.62 () {58 vars in 1442 bytes} [Wed Aug 24 05:57:22 2022] PUT /identity/v3/projects/9391bbe626b44799942ef25c41bc33ad/groups/cec1f56499194c9fb4ba718917f95478/roles/580aa7dbb38d4cff8f9d1bb8983f99ef => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:57:22.719963 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72860763-55fe-443b-8bc1-ebbda64883e8 None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:22.846035 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72860763-55fe-443b-8bc1-ebbda64883e8 None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:22.886588 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768bed11-e71d-4470-8916-8de04f3e7cab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:22.887643 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768bed11-e71d-4470-8916-8de04f3e7cab None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:22.888135 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768bed11-e71d-4470-8916-8de04f3e7cab None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:22.888669 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768bed11-e71d-4470-8916-8de04f3e7cab None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:22.902000 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768bed11-e71d-4470-8916-8de04f3e7cab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:57:22.907200 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application [None req-768bed11-e71d-4470-8916-8de04f3e7cab None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 05:57:22.907200 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:57:22.907200 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:57:22.907200 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:57:22.907200 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:57:22.907200 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:57:22.907200 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:57:22.907200 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:57:22.907200 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:57:22.907200 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:57:22.907200 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:57:22.907200 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:57:22.907200 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:57:22.907200 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:57:22.907200 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:57:22.907200 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:57:22.908187 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:57:22.908187 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:57:22.908187 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:57:22.908187 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:57:22.908187 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:57:22.908187 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:57:22.908187 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:57:22.908187 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:57:22.908187 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:57:22.908187 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:57:22.908187 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:57:22.908187 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application with Lock( Aug 24 05:57:22.908187 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:57:22.908187 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:57:22.908187 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:57:22.908187 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:57:22.909093 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:57:22.909093 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:57:22.909093 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:57:22.909093 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:57:22.909093 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:57:22.909093 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:57:22.909093 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:57:22.909093 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:57:22.909093 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:57:22.909093 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:57:22.909093 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 05:57:22.909093 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Aug 24 05:57:22.916452 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 201/399] 10.222.0.62 () {56 vars in 1114 bytes} [Wed Aug 24 05:57:22 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:57:22.917493 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443ccfed-f6d4-4bb4-913e-e7c65b865146 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:22.937214 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768bed11-e71d-4470-8916-8de04f3e7cab None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:22.947286 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:22.958641 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:23.069780 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443ccfed-f6d4-4bb4-913e-e7c65b865146 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:23.086007 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768bed11-e71d-4470-8916-8de04f3e7cab None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:23.109969 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5f9669-86cb-4370-8240-01d0a5707f62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:23.111933 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5f9669-86cb-4370-8240-01d0a5707f62 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:23.112381 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5f9669-86cb-4370-8240-01d0a5707f62 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:23.113105 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5f9669-86cb-4370-8240-01d0a5707f62 None admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:23.113937 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5f9669-86cb-4370-8240-01d0a5707f62 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:57:23.114650 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5f9669-86cb-4370-8240-01d0a5707f62 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:23.116934 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5f9669-86cb-4370-8240-01d0a5707f62 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:23.127901 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffae353-9060-445b-ad3f-f5b5e3abd1f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:23.128627 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffae353-9060-445b-ad3f-f5b5e3abd1f2 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:23.128832 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffae353-9060-445b-ad3f-f5b5e3abd1f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:23.129113 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffae353-9060-445b-ad3f-f5b5e3abd1f2 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:23.133620 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 199/400] 10.222.0.62 () {56 vars in 1111 bytes} [Wed Aug 24 05:57:22 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:57:23.133900 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153f57a2-2063-4494-ad50-90201d961446 None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:23.134179 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153f57a2-2063-4494-ad50-90201d961446 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:23.134448 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153f57a2-2063-4494-ad50-90201d961446 None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:23.135818 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 200/401] 10.222.0.62 () {54 vars in 927 bytes} [Wed Aug 24 05:57:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:57:23.136370 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153f57a2-2063-4494-ad50-90201d961446 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:23.148881 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffae353-9060-445b-ad3f-f5b5e3abd1f2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:57:23.155327 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application [None req-4ffae353-9060-445b-ad3f-f5b5e3abd1f2 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 05:57:23.155327 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:57:23.155327 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:57:23.155327 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:57:23.155327 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:57:23.155327 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:57:23.155327 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:57:23.155327 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:57:23.155327 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:57:23.155327 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:57:23.155327 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:57:23.155327 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:57:23.155327 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:57:23.155327 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:57:23.155327 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:57:23.155327 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:57:23.156232 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:57:23.156232 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:57:23.156232 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:57:23.156232 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:57:23.156232 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:57:23.156232 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:57:23.156232 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:57:23.156232 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:57:23.156232 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:57:23.156232 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:57:23.156232 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:57:23.156232 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application with Lock( Aug 24 05:57:23.156232 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:57:23.156232 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:57:23.156232 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:57:23.156232 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:57:23.157077 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:57:23.157077 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:57:23.157077 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:57:23.157077 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:57:23.157077 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:57:23.157077 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:57:23.157077 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:57:23.157077 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:57:23.157077 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:57:23.157077 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:57:23.157077 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 05:57:23.157077 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Aug 24 05:57:23.157077 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 202/402] 10.222.0.62 () {56 vars in 1114 bytes} [Wed Aug 24 05:57:22 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 05:57:23.158799 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f0ed2c-b3dc-4683-9be0-56c8dc819795 None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:23.158933 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f0ed2c-b3dc-4683-9be0-56c8dc819795 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:23.159182 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f0ed2c-b3dc-4683-9be0-56c8dc819795 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:23.184009 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-c6f0ed2c-b3dc-4683-9be0-56c8dc819795 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:57:23.185348 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-c6f0ed2c-b3dc-4683-9be0-56c8dc819795 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:57:23.232633 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:23.255424 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153f57a2-2063-4494-ad50-90201d961446 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:23.265861 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:23.273150 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 203/403] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Aug 24 05:57:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 05:57:23.274405 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f0ed2c-b3dc-4683-9be0-56c8dc819795 None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:23.290762 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbf4e48-c6f2-482e-82e0-75c26a58126f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:23.291404 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbf4e48-c6f2-482e-82e0-75c26a58126f None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:23.291685 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbf4e48-c6f2-482e-82e0-75c26a58126f None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:23.292296 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbf4e48-c6f2-482e-82e0-75c26a58126f None admin] PATH_INFO: `/v3/groups/cec1f56499194c9fb4ba718917f95478` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:23.293096 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbf4e48-c6f2-482e-82e0-75c26a58126f None admin] RBAC: Authorizing `identity:get_group(group_id=cec1f56499194c9fb4ba718917f95478)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:23.294559 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbf4e48-c6f2-482e-82e0-75c26a58126f None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:23.296805 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 201/404] 10.222.0.62 () {56 vars in 1180 bytes} [Wed Aug 24 05:57:23 2022] GET /identity/v3/groups/cec1f56499194c9fb4ba718917f95478 => generated 226 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:57:23.302173 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbf4e48-c6f2-482e-82e0-75c26a58126f None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:23.390594 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f0ed2c-b3dc-4683-9be0-56c8dc819795 None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:23.406420 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbf4e48-c6f2-482e-82e0-75c26a58126f None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:23.452138 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba717724-2cb7-4878-b578-d450af7a2823 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:23.452778 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba717724-2cb7-4878-b578-d450af7a2823 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:23.453059 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba717724-2cb7-4878-b578-d450af7a2823 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:23.455009 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba717724-2cb7-4878-b578-d450af7a2823 None admin] PATH_INFO: `/v3/roles` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:23.455678 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba717724-2cb7-4878-b578-d450af7a2823 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:57:23.455981 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba717724-2cb7-4878-b578-d450af7a2823 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:23.458279 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba717724-2cb7-4878-b578-d450af7a2823 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:23.467350 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613c156e-a92b-4593-9dc0-3a0b6472e124 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:23.468042 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613c156e-a92b-4593-9dc0-3a0b6472e124 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:23.468326 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613c156e-a92b-4593-9dc0-3a0b6472e124 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:23.468625 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613c156e-a92b-4593-9dc0-3a0b6472e124 None admin] PATH_INFO: `/v3/projects/6d7fd5ca249a4df5aba301beb23e2eb7` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:23.469782 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613c156e-a92b-4593-9dc0-3a0b6472e124 None admin] RBAC: Authorizing `identity:get_project(project_id=6d7fd5ca249a4df5aba301beb23e2eb7)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:23.471228 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613c156e-a92b-4593-9dc0-3a0b6472e124 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:23.473262 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 202/405] 10.222.0.62 () {56 vars in 1186 bytes} [Wed Aug 24 05:57:23 2022] GET /identity/v3/projects/6d7fd5ca249a4df5aba301beb23e2eb7 => generated 298 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:57:23.481625 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613c156e-a92b-4593-9dc0-3a0b6472e124 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:23.484632 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 204/406] 10.222.0.62 () {56 vars in 1111 bytes} [Wed Aug 24 05:57:23 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:57:23.496345 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba717724-2cb7-4878-b578-d450af7a2823 None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:23.505423 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:23.601373 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:23.654716 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba717724-2cb7-4878-b578-d450af7a2823 None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:23.661326 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613c156e-a92b-4593-9dc0-3a0b6472e124 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:23.699302 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265f4a1d-b9a8-4206-a211-6a690f4c286a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:23.699934 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265f4a1d-b9a8-4206-a211-6a690f4c286a None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:23.700212 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265f4a1d-b9a8-4206-a211-6a690f4c286a None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:23.700481 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265f4a1d-b9a8-4206-a211-6a690f4c286a None admin] PATH_INFO: `/v3/groups/cec1f56499194c9fb4ba718917f95478` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:23.702350 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265f4a1d-b9a8-4206-a211-6a690f4c286a None admin] RBAC: Authorizing `identity:get_group(group_id=cec1f56499194c9fb4ba718917f95478)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:23.705282 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265f4a1d-b9a8-4206-a211-6a690f4c286a None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:23.708567 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 205/407] 10.222.0.62 () {56 vars in 1180 bytes} [Wed Aug 24 05:57:23 2022] GET /identity/v3/groups/cec1f56499194c9fb4ba718917f95478 => generated 226 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:57:23.709637 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265f4a1d-b9a8-4206-a211-6a690f4c286a None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:23.720380 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c562704-39aa-4c1a-8648-069ebfa555a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:23.721084 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c562704-39aa-4c1a-8648-069ebfa555a3 None admin] REQUEST_METHOD: `PUT` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:23.721406 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c562704-39aa-4c1a-8648-069ebfa555a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:23.721688 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c562704-39aa-4c1a-8648-069ebfa555a3 None admin] PATH_INFO: `/v3/projects/6d7fd5ca249a4df5aba301beb23e2eb7/groups/cec1f56499194c9fb4ba718917f95478/roles/ba3c2c3cb98c424f9ece19903a59c53c` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:23.725251 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c562704-39aa-4c1a-8648-069ebfa555a3 None admin] RBAC: Authorizing `identity:create_grant(project_id=6d7fd5ca249a4df5aba301beb23e2eb7, group_id=cec1f56499194c9fb4ba718917f95478, role_id=ba3c2c3cb98c424f9ece19903a59c53c)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:23.726803 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c562704-39aa-4c1a-8648-069ebfa555a3 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:23.820754 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265f4a1d-b9a8-4206-a211-6a690f4c286a None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:23.831790 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 203/408] 10.222.0.62 () {58 vars in 1442 bytes} [Wed Aug 24 05:57:23 2022] PUT /identity/v3/projects/6d7fd5ca249a4df5aba301beb23e2eb7/groups/cec1f56499194c9fb4ba718917f95478/roles/ba3c2c3cb98c424f9ece19903a59c53c => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:57:23.833664 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c562704-39aa-4c1a-8648-069ebfa555a3 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:23.862930 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:23.870418 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb63f4b7-f752-44bd-b7f7-8ba8c4cdbed5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:23.871568 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb63f4b7-f752-44bd-b7f7-8ba8c4cdbed5 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:23.871856 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb63f4b7-f752-44bd-b7f7-8ba8c4cdbed5 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:23.872137 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb63f4b7-f752-44bd-b7f7-8ba8c4cdbed5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:23.887495 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb63f4b7-f752-44bd-b7f7-8ba8c4cdbed5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:57:23.894309 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application [None req-eb63f4b7-f752-44bd-b7f7-8ba8c4cdbed5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 05:57:23.894309 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:57:23.894309 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:57:23.894309 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:57:23.894309 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:57:23.894309 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:57:23.894309 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:57:23.894309 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:57:23.894309 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:57:23.894309 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:57:23.894309 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:57:23.894309 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:57:23.894309 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:57:23.894309 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:57:23.894309 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:57:23.894309 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:57:23.895701 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:57:23.895701 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:57:23.895701 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:57:23.895701 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:57:23.895701 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:57:23.895701 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:57:23.895701 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:57:23.895701 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:57:23.895701 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:57:23.895701 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:57:23.895701 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:57:23.895701 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application with Lock( Aug 24 05:57:23.895701 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:57:23.895701 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:57:23.895701 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:57:23.895701 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:57:23.896810 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:57:23.896810 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:57:23.896810 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:57:23.896810 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:57:23.896810 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:57:23.896810 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:57:23.896810 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:57:23.896810 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:57:23.896810 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:57:23.896810 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:57:23.896810 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 05:57:23.896810 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Aug 24 05:57:23.896810 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 206/409] 10.222.0.62 () {56 vars in 1096 bytes} [Wed Aug 24 05:57:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:57:23.902296 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb63f4b7-f752-44bd-b7f7-8ba8c4cdbed5 None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:23.991965 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:24.040568 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c562704-39aa-4c1a-8648-069ebfa555a3 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:24.087818 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb63f4b7-f752-44bd-b7f7-8ba8c4cdbed5 None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:24.093745 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954cf3c8-e69f-4ad2-84a5-07d8e438039c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:24.095523 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954cf3c8-e69f-4ad2-84a5-07d8e438039c None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:24.096012 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954cf3c8-e69f-4ad2-84a5-07d8e438039c None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:24.096484 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954cf3c8-e69f-4ad2-84a5-07d8e438039c None admin] PATH_INFO: `/v3/projects/9391bbe626b44799942ef25c41bc33ad` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:24.097928 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954cf3c8-e69f-4ad2-84a5-07d8e438039c None admin] RBAC: Authorizing `identity:get_project(project_id=9391bbe626b44799942ef25c41bc33ad)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:24.099747 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954cf3c8-e69f-4ad2-84a5-07d8e438039c None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:24.102207 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 204/410] 10.222.0.62 () {56 vars in 1186 bytes} [Wed Aug 24 05:57:23 2022] GET /identity/v3/projects/9391bbe626b44799942ef25c41bc33ad => generated 302 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:57:24.108535 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954cf3c8-e69f-4ad2-84a5-07d8e438039c None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:24.161494 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892e3813-7314-463e-bc37-42d25e662df2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:24.162541 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892e3813-7314-463e-bc37-42d25e662df2 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:24.163801 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892e3813-7314-463e-bc37-42d25e662df2 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:24.164277 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892e3813-7314-463e-bc37-42d25e662df2 None admin] PATH_INFO: `/v3/roles` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:24.168721 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892e3813-7314-463e-bc37-42d25e662df2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:57:24.169260 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892e3813-7314-463e-bc37-42d25e662df2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:24.171953 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892e3813-7314-463e-bc37-42d25e662df2 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:24.190806 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 207/411] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Aug 24 05:57:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:57:24.197159 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892e3813-7314-463e-bc37-42d25e662df2 None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:24.261174 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954cf3c8-e69f-4ad2-84a5-07d8e438039c None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:24.274347 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:24.298440 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c368acec-7bac-42de-96b8-572980b5a472 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:24.299368 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c368acec-7bac-42de-96b8-572980b5a472 None admin] REQUEST_METHOD: `PUT` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:24.299789 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c368acec-7bac-42de-96b8-572980b5a472 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:24.300213 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c368acec-7bac-42de-96b8-572980b5a472 None admin] PATH_INFO: `/v3/projects/9391bbe626b44799942ef25c41bc33ad/groups/cec1f56499194c9fb4ba718917f95478/roles/ba3c2c3cb98c424f9ece19903a59c53c` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:24.302619 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c368acec-7bac-42de-96b8-572980b5a472 None admin] RBAC: Authorizing `identity:create_grant(project_id=9391bbe626b44799942ef25c41bc33ad, group_id=cec1f56499194c9fb4ba718917f95478, role_id=ba3c2c3cb98c424f9ece19903a59c53c)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:24.304232 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c368acec-7bac-42de-96b8-572980b5a472 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:24.341896 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892e3813-7314-463e-bc37-42d25e662df2 None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:24.377146 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 205/412] 10.222.0.62 () {58 vars in 1442 bytes} [Wed Aug 24 05:57:24 2022] PUT /identity/v3/projects/9391bbe626b44799942ef25c41bc33ad/groups/cec1f56499194c9fb4ba718917f95478/roles/ba3c2c3cb98c424f9ece19903a59c53c => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 05:57:24.378721 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8140c86-e8db-4849-bae8-cad8c2b988d4 None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:24.379111 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8140c86-e8db-4849-bae8-cad8c2b988d4 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:24.379389 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8140c86-e8db-4849-bae8-cad8c2b988d4 None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:24.385200 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 206/413] 10.222.0.62 () {54 vars in 927 bytes} [Wed Aug 24 05:57:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:57:24.385388 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d73c8de-dc40-4468-8a45-4eb5a4d88f22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:24.386104 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d73c8de-dc40-4468-8a45-4eb5a4d88f22 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:24.386390 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d73c8de-dc40-4468-8a45-4eb5a4d88f22 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:24.386693 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d73c8de-dc40-4468-8a45-4eb5a4d88f22 None admin] PATH_INFO: `/v3/groups/cbf8f7f1550c4404831928e6ad3a3393` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:24.392574 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3773bb0a-f56b-442c-9b6c-1afde0a9122c None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:24.392574 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3773bb0a-f56b-442c-9b6c-1afde0a9122c None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:24.392574 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3773bb0a-f56b-442c-9b6c-1afde0a9122c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:24.393420 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d73c8de-dc40-4468-8a45-4eb5a4d88f22 None admin] RBAC: Authorizing `identity:get_group(group_id=cbf8f7f1550c4404831928e6ad3a3393)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:24.394916 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d73c8de-dc40-4468-8a45-4eb5a4d88f22 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:24.404179 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 208/414] 10.222.0.62 () {56 vars in 1180 bytes} [Wed Aug 24 05:57:24 2022] GET /identity/v3/groups/cbf8f7f1550c4404831928e6ad3a3393 => generated 229 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:57:24.405191 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d73c8de-dc40-4468-8a45-4eb5a4d88f22 None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:24.408434 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:24.429775 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-3773bb0a-f56b-442c-9b6c-1afde0a9122c None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:57:24.431844 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-3773bb0a-f56b-442c-9b6c-1afde0a9122c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:57:24.517523 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 207/415] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Aug 24 05:57:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 05:57:24.594634 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d73c8de-dc40-4468-8a45-4eb5a4d88f22 None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:24.616103 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:24.633271 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c31f9a-1ef4-47ba-9cb7-5e0a25f836d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:24.634250 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c31f9a-1ef4-47ba-9cb7-5e0a25f836d5 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:24.634740 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c31f9a-1ef4-47ba-9cb7-5e0a25f836d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:24.635202 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c31f9a-1ef4-47ba-9cb7-5e0a25f836d5 None admin] PATH_INFO: `/v3/projects/632e93ce840f4853b64b4d659536e045` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:24.636560 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c31f9a-1ef4-47ba-9cb7-5e0a25f836d5 None admin] RBAC: Authorizing `identity:get_project(project_id=632e93ce840f4853b64b4d659536e045)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:24.638229 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c31f9a-1ef4-47ba-9cb7-5e0a25f836d5 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:24.640159 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 209/416] 10.222.0.62 () {56 vars in 1186 bytes} [Wed Aug 24 05:57:24 2022] GET /identity/v3/projects/632e93ce840f4853b64b4d659536e045 => generated 344 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:57:24.646062 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3773bb0a-f56b-442c-9b6c-1afde0a9122c None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:24.771029 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3773bb0a-f56b-442c-9b6c-1afde0a9122c None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:24.814859 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:24.821401 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63a632f-d64d-4cd6-bb6a-b06c96eb36ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:24.822073 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63a632f-d64d-4cd6-bb6a-b06c96eb36ec None admin] REQUEST_METHOD: `PUT` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:24.822352 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63a632f-d64d-4cd6-bb6a-b06c96eb36ec None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:24.822628 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63a632f-d64d-4cd6-bb6a-b06c96eb36ec None admin] PATH_INFO: `/v3/projects/632e93ce840f4853b64b4d659536e045/groups/cbf8f7f1550c4404831928e6ad3a3393/roles/49e47cd157d743f6905f1db07ec878a7` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:24.827070 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c31f9a-1ef4-47ba-9cb7-5e0a25f836d5 None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:24.829842 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63a632f-d64d-4cd6-bb6a-b06c96eb36ec None admin] RBAC: Authorizing `identity:create_grant(project_id=632e93ce840f4853b64b4d659536e045, group_id=cbf8f7f1550c4404831928e6ad3a3393, role_id=49e47cd157d743f6905f1db07ec878a7)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:24.831301 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63a632f-d64d-4cd6-bb6a-b06c96eb36ec None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:24.923467 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 208/417] 10.222.0.62 () {58 vars in 1442 bytes} [Wed Aug 24 05:57:24 2022] PUT /identity/v3/projects/632e93ce840f4853b64b4d659536e045/groups/cbf8f7f1550c4404831928e6ad3a3393/roles/49e47cd157d743f6905f1db07ec878a7 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 05:57:24.946430 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c31f9a-1ef4-47ba-9cb7-5e0a25f836d5 None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:24.984355 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f543ea-07da-403c-8c73-fef4baee21b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:24.985098 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f543ea-07da-403c-8c73-fef4baee21b7 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:24.985682 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f543ea-07da-403c-8c73-fef4baee21b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:24.985956 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f543ea-07da-403c-8c73-fef4baee21b7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:24.993307 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e158069-7553-4ffe-9814-b1899b162963 None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:24.993923 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e158069-7553-4ffe-9814-b1899b162963 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:24.994400 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e158069-7553-4ffe-9814-b1899b162963 None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:24.996189 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 209/418] 10.222.0.62 () {54 vars in 927 bytes} [Wed Aug 24 05:57:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 05:57:24.999930 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f543ea-07da-403c-8c73-fef4baee21b7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:57:25.002332 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0844d6a4-4c86-430f-9c16-f24d9a24c47d None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:25.002852 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0844d6a4-4c86-430f-9c16-f24d9a24c47d None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:25.003314 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0844d6a4-4c86-430f-9c16-f24d9a24c47d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:25.004652 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application [None req-05f543ea-07da-403c-8c73-fef4baee21b7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 05:57:25.004652 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:57:25.004652 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:57:25.004652 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:57:25.004652 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:57:25.004652 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:57:25.004652 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:57:25.004652 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:57:25.004652 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:57:25.004652 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:57:25.004652 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:57:25.004652 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:57:25.004652 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:57:25.004652 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:57:25.004652 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:57:25.004652 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:57:25.005540 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:57:25.005540 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:57:25.005540 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:57:25.005540 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:57:25.005540 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:57:25.005540 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:57:25.005540 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:57:25.005540 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:57:25.005540 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:57:25.005540 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:57:25.005540 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:57:25.005540 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application with Lock( Aug 24 05:57:25.005540 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:57:25.005540 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:57:25.005540 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:57:25.005540 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:57:25.006307 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:57:25.006307 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:57:25.006307 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:57:25.006307 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:57:25.006307 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:57:25.006307 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:57:25.006307 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:57:25.006307 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:57:25.006307 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:57:25.006307 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:57:25.006307 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 05:57:25.006307 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Aug 24 05:57:25.006307 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 210/419] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Aug 24 05:57:24 2022] GET /identity/v3/roles/member => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 05:57:25.015136 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f543ea-07da-403c-8c73-fef4baee21b7 None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:25.049444 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-0844d6a4-4c86-430f-9c16-f24d9a24c47d None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:57:25.053057 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-0844d6a4-4c86-430f-9c16-f24d9a24c47d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:57:25.122126 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:25.178512 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 210/420] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Aug 24 05:57:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 05:57:25.189265 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f543ea-07da-403c-8c73-fef4baee21b7 None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:25.226028 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3149ba7e-004a-4637-bc93-91d5af060e18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:25.226821 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3149ba7e-004a-4637-bc93-91d5af060e18 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:25.227195 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3149ba7e-004a-4637-bc93-91d5af060e18 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:25.227562 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3149ba7e-004a-4637-bc93-91d5af060e18 None admin] PATH_INFO: `/v3/roles` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:25.228396 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3149ba7e-004a-4637-bc93-91d5af060e18 None admin] RBAC: Adding query filter params (name=member) {{(pid=81851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:57:25.228927 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3149ba7e-004a-4637-bc93-91d5af060e18 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:25.230453 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3149ba7e-004a-4637-bc93-91d5af060e18 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:25.240237 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 211/421] 10.222.0.62 () {56 vars in 1101 bytes} [Wed Aug 24 05:57:25 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:57:25.245746 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0844d6a4-4c86-430f-9c16-f24d9a24c47d None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:25.384788 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0844d6a4-4c86-430f-9c16-f24d9a24c47d None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:25.409479 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:25.421452 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3a394e-07e7-4ec7-9849-cb1cc7093264 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:25.424165 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3a394e-07e7-4ec7-9849-cb1cc7093264 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:25.424165 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3a394e-07e7-4ec7-9849-cb1cc7093264 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:25.424165 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3a394e-07e7-4ec7-9849-cb1cc7093264 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:25.492520 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3a394e-07e7-4ec7-9849-cb1cc7093264 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:25.494039 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3a394e-07e7-4ec7-9849-cb1cc7093264 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:25.597321 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 211/422] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Aug 24 05:57:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 05:57:25.603093 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3149ba7e-004a-4637-bc93-91d5af060e18 None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:25.664740 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3a394e-07e7-4ec7-9849-cb1cc7093264 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:25.695806 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:25.780771 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3a394e-07e7-4ec7-9849-cb1cc7093264 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:25.807262 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3149ba7e-004a-4637-bc93-91d5af060e18 None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:25.817289 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9e2d37-9b25-48c8-9f05-0f509ca3b7b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:25.817923 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9e2d37-9b25-48c8-9f05-0f509ca3b7b0 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:25.818201 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9e2d37-9b25-48c8-9f05-0f509ca3b7b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:25.818471 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9e2d37-9b25-48c8-9f05-0f509ca3b7b0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:25.828392 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:25.831666 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9e2d37-9b25-48c8-9f05-0f509ca3b7b0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:57:25.836581 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-af9e2d37-9b25-48c8-9f05-0f509ca3b7b0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 05:57:25.836581 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:57:25.836581 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:57:25.836581 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:57:25.836581 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:57:25.836581 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:57:25.836581 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:57:25.836581 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:57:25.836581 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:57:25.836581 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:57:25.836581 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:57:25.836581 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:57:25.836581 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:57:25.836581 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:57:25.836581 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:57:25.836581 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:57:25.837465 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:57:25.837465 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:57:25.837465 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:57:25.837465 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:57:25.837465 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:57:25.837465 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:57:25.837465 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:57:25.837465 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:57:25.837465 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:57:25.837465 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:57:25.837465 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:57:25.837465 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 24 05:57:25.837465 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:57:25.837465 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:57:25.837465 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:57:25.837465 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:57:25.838830 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:57:25.838830 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:57:25.838830 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:57:25.838830 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:57:25.838830 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:57:25.838830 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:57:25.838830 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:57:25.838830 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:57:25.838830 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:57:25.838830 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:57:25.838830 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 05:57:25.838830 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 24 05:57:25.838830 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 212/423] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Aug 24 05:57:25 2022] GET /identity/v3/roles/member => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 05:57:25.843812 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9e2d37-9b25-48c8-9f05-0f509ca3b7b0 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:25.907125 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82237aab-53a7-4e1d-9bbe-9eb6583dd933 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:25.908258 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82237aab-53a7-4e1d-9bbe-9eb6583dd933 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:25.908561 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82237aab-53a7-4e1d-9bbe-9eb6583dd933 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:25.908840 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82237aab-53a7-4e1d-9bbe-9eb6583dd933 None admin] PATH_INFO: `/v3/projects/6d7fd5ca249a4df5aba301beb23e2eb7` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:25.909965 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82237aab-53a7-4e1d-9bbe-9eb6583dd933 None admin] RBAC: Authorizing `identity:get_project(project_id=6d7fd5ca249a4df5aba301beb23e2eb7)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:25.911456 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82237aab-53a7-4e1d-9bbe-9eb6583dd933 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:25.913406 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 212/424] 10.222.0.62 () {56 vars in 1186 bytes} [Wed Aug 24 05:57:25 2022] GET /identity/v3/projects/6d7fd5ca249a4df5aba301beb23e2eb7 => generated 298 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:57:25.925445 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82237aab-53a7-4e1d-9bbe-9eb6583dd933 None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:25.939496 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:25.956873 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9e2d37-9b25-48c8-9f05-0f509ca3b7b0 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:25.992369 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85b3ede-bad8-4703-aa7c-f05a84ad162d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:25.993006 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85b3ede-bad8-4703-aa7c-f05a84ad162d None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:25.993294 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85b3ede-bad8-4703-aa7c-f05a84ad162d None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:25.993575 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85b3ede-bad8-4703-aa7c-f05a84ad162d None admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:25.994238 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85b3ede-bad8-4703-aa7c-f05a84ad162d None admin] RBAC: Adding query filter params (name=member) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:57:25.994547 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85b3ede-bad8-4703-aa7c-f05a84ad162d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:25.996265 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85b3ede-bad8-4703-aa7c-f05a84ad162d None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:26.005109 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 213/425] 10.222.0.62 () {56 vars in 1101 bytes} [Wed Aug 24 05:57:25 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:57:26.010523 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85b3ede-bad8-4703-aa7c-f05a84ad162d None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:26.115585 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85b3ede-bad8-4703-aa7c-f05a84ad162d None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:26.139150 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82237aab-53a7-4e1d-9bbe-9eb6583dd933 None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:26.150838 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af83aca1-80d6-4a99-aee0-10b137377be8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:26.151466 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af83aca1-80d6-4a99-aee0-10b137377be8 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:26.151753 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af83aca1-80d6-4a99-aee0-10b137377be8 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:26.152035 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af83aca1-80d6-4a99-aee0-10b137377be8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:26.179450 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:26.226619 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af83aca1-80d6-4a99-aee0-10b137377be8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:26.228136 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af83aca1-80d6-4a99-aee0-10b137377be8 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:26.229805 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31364f2-2681-4acb-842a-fb6a89bdd7b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:26.230550 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31364f2-2681-4acb-842a-fb6a89bdd7b4 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:26.231654 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31364f2-2681-4acb-842a-fb6a89bdd7b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:26.231654 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31364f2-2681-4acb-842a-fb6a89bdd7b4 None admin] PATH_INFO: `/v3/groups/cec1f56499194c9fb4ba718917f95478` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:26.233746 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31364f2-2681-4acb-842a-fb6a89bdd7b4 None admin] RBAC: Authorizing `identity:get_group(group_id=cec1f56499194c9fb4ba718917f95478)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:26.235905 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31364f2-2681-4acb-842a-fb6a89bdd7b4 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:26.239214 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 213/426] 10.222.0.62 () {56 vars in 1180 bytes} [Wed Aug 24 05:57:25 2022] GET /identity/v3/groups/cec1f56499194c9fb4ba718917f95478 => generated 226 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:57:26.241133 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e620bd1-3885-4711-af96-280a032057f3 None None] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:26.241549 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e620bd1-3885-4711-af96-280a032057f3 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:26.241943 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e620bd1-3885-4711-af96-280a032057f3 None None] PATH_INFO: `/` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:26.243286 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 214/427] 10.222.0.62 () {54 vars in 927 bytes} [Wed Aug 24 05:57:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:57:26.250161 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e620bd1-3885-4711-af96-280a032057f3 None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:26.309777 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:26.350431 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 214/428] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Aug 24 05:57:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 24 05:57:26.352147 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c446b214-9e53-4044-937d-b528c3ef4f58 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:26.352661 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c446b214-9e53-4044-937d-b528c3ef4f58 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:26.353078 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c446b214-9e53-4044-937d-b528c3ef4f58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:26.371144 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-c446b214-9e53-4044-937d-b528c3ef4f58 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:57:26.372619 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-c446b214-9e53-4044-937d-b528c3ef4f58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:57:26.378755 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e620bd1-3885-4711-af96-280a032057f3 None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:26.416782 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27bd07f-e3b0-4e19-810c-4386958292cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:26.417619 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:26.418540 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27bd07f-e3b0-4e19-810c-4386958292cb None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:26.419061 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27bd07f-e3b0-4e19-810c-4386958292cb None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:26.419537 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27bd07f-e3b0-4e19-810c-4386958292cb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:26.420857 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27bd07f-e3b0-4e19-810c-4386958292cb None admin] RBAC: Adding query filter params (group.id=cec1f56499194c9fb4ba718917f95478, role.id=580aa7dbb38d4cff8f9d1bb8983f99ef, scope.project.id=6d7fd5ca249a4df5aba301beb23e2eb7) {{(pid=81851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:57:26.421417 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27bd07f-e3b0-4e19-810c-4386958292cb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:26.423178 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27bd07f-e3b0-4e19-810c-4386958292cb None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:26.436828 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 215/429] 10.222.0.62 () {56 vars in 1376 bytes} [Wed Aug 24 05:57:26 2022] GET /identity/v3/role_assignments?group.id=cec1f56499194c9fb4ba718917f95478&scope.project.id=6d7fd5ca249a4df5aba301beb23e2eb7&role.id=580aa7dbb38d4cff8f9d1bb8983f99ef => generated 611 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:57:26.438162 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27bd07f-e3b0-4e19-810c-4386958292cb None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:26.452727 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 215/430] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Aug 24 05:57:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 05:57:26.581347 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27bd07f-e3b0-4e19-810c-4386958292cb None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:26.593317 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:26.620357 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00559e00-6274-428f-953a-49ec44e34364 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:26.621478 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00559e00-6274-428f-953a-49ec44e34364 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:26.622014 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00559e00-6274-428f-953a-49ec44e34364 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:26.622505 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00559e00-6274-428f-953a-49ec44e34364 None admin] PATH_INFO: `/v3/projects/9391bbe626b44799942ef25c41bc33ad` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:26.623843 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00559e00-6274-428f-953a-49ec44e34364 None admin] RBAC: Authorizing `identity:get_project(project_id=9391bbe626b44799942ef25c41bc33ad)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:26.625618 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00559e00-6274-428f-953a-49ec44e34364 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:26.628143 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 216/431] 10.222.0.62 () {56 vars in 1186 bytes} [Wed Aug 24 05:57:26 2022] GET /identity/v3/projects/9391bbe626b44799942ef25c41bc33ad => generated 302 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:57:26.634000 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c446b214-9e53-4044-937d-b528c3ef4f58 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:26.748117 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00559e00-6274-428f-953a-49ec44e34364 None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:26.779996 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c446b214-9e53-4044-937d-b528c3ef4f58 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:26.826259 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767e5c3c-87e7-419f-aba2-d63f27452678 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:26.826897 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767e5c3c-87e7-419f-aba2-d63f27452678 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:26.827181 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767e5c3c-87e7-419f-aba2-d63f27452678 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:26.827451 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767e5c3c-87e7-419f-aba2-d63f27452678 None admin] PATH_INFO: `/v3/groups/cec1f56499194c9fb4ba718917f95478` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:26.828659 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767e5c3c-87e7-419f-aba2-d63f27452678 None admin] RBAC: Authorizing `identity:get_group(group_id=cec1f56499194c9fb4ba718917f95478)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:26.830622 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767e5c3c-87e7-419f-aba2-d63f27452678 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:26.832546 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 216/432] 10.222.0.62 () {56 vars in 1180 bytes} [Wed Aug 24 05:57:26 2022] GET /identity/v3/groups/cec1f56499194c9fb4ba718917f95478 => generated 226 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:57:26.833727 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ac07aa-b25c-4d8b-afc5-bdc6adba4f8d None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:26.834004 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ac07aa-b25c-4d8b-afc5-bdc6adba4f8d None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:26.834272 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ac07aa-b25c-4d8b-afc5-bdc6adba4f8d None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:26.835680 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 217/433] 10.222.0.62 () {54 vars in 927 bytes} [Wed Aug 24 05:57:26 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:57:26.840394 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ac07aa-b25c-4d8b-afc5-bdc6adba4f8d None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:26.879426 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00559e00-6274-428f-953a-49ec44e34364 None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:26.887391 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:26.919016 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a527ffe-ffbd-4f20-8783-1e5d53c63374 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:26.919735 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a527ffe-ffbd-4f20-8783-1e5d53c63374 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:26.920079 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a527ffe-ffbd-4f20-8783-1e5d53c63374 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:26.920690 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a527ffe-ffbd-4f20-8783-1e5d53c63374 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:26.932931 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a527ffe-ffbd-4f20-8783-1e5d53c63374 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:57:26.939462 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application [None req-5a527ffe-ffbd-4f20-8783-1e5d53c63374 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 05:57:26.939462 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:57:26.939462 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:57:26.939462 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:57:26.939462 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:57:26.939462 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:57:26.939462 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:57:26.939462 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:57:26.939462 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:57:26.939462 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:57:26.939462 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:57:26.939462 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:57:26.939462 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:57:26.939462 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:57:26.939462 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:57:26.939462 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:57:26.940365 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:57:26.940365 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:57:26.940365 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:57:26.940365 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:57:26.940365 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:57:26.940365 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:57:26.940365 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:57:26.940365 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:57:26.940365 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:57:26.940365 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:57:26.940365 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:57:26.940365 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application with Lock( Aug 24 05:57:26.940365 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:57:26.940365 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:57:26.940365 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:57:26.940365 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:57:26.941219 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:57:26.941219 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:57:26.941219 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:57:26.941219 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:57:26.941219 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:57:26.941219 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:57:26.941219 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:57:26.941219 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:57:26.941219 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:57:26.941219 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:57:26.941219 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 05:57:26.941219 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Aug 24 05:57:26.941219 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 217/434] 10.222.0.62 () {56 vars in 1114 bytes} [Wed Aug 24 05:57:26 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 05:57:26.942399 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b630e61d-dc0e-4bdd-9014-b40be5f4c4a7 None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:26.942399 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b630e61d-dc0e-4bdd-9014-b40be5f4c4a7 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:26.942567 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b630e61d-dc0e-4bdd-9014-b40be5f4c4a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:26.962739 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-b630e61d-dc0e-4bdd-9014-b40be5f4c4a7 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:57:26.963946 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-b630e61d-dc0e-4bdd-9014-b40be5f4c4a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:57:26.971401 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ac07aa-b25c-4d8b-afc5-bdc6adba4f8d None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:27.007467 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:27.018869 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31dd4c7e-066c-4156-b497-0ff0002ad062 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:27.020880 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dd4c7e-066c-4156-b497-0ff0002ad062 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:27.021227 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dd4c7e-066c-4156-b497-0ff0002ad062 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:27.021512 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dd4c7e-066c-4156-b497-0ff0002ad062 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:27.026639 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31dd4c7e-066c-4156-b497-0ff0002ad062 None admin] RBAC: Adding query filter params (group.id=cec1f56499194c9fb4ba718917f95478, role.id=580aa7dbb38d4cff8f9d1bb8983f99ef, scope.project.id=9391bbe626b44799942ef25c41bc33ad) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:57:27.026639 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31dd4c7e-066c-4156-b497-0ff0002ad062 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:27.026639 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31dd4c7e-066c-4156-b497-0ff0002ad062 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:27.037275 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 218/435] 10.222.0.62 () {56 vars in 1376 bytes} [Wed Aug 24 05:57:26 2022] GET /identity/v3/role_assignments?group.id=cec1f56499194c9fb4ba718917f95478&scope.project.id=9391bbe626b44799942ef25c41bc33ad&role.id=580aa7dbb38d4cff8f9d1bb8983f99ef => generated 611 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:57:27.038297 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31dd4c7e-066c-4156-b497-0ff0002ad062 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:27.061679 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 218/436] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Aug 24 05:57:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 05:57:27.164275 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31dd4c7e-066c-4156-b497-0ff0002ad062 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:27.172569 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:27.202887 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845f09e2-112d-424a-88d0-e86ae1f54e9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:27.203499 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845f09e2-112d-424a-88d0-e86ae1f54e9f None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:27.203750 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845f09e2-112d-424a-88d0-e86ae1f54e9f None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:27.204087 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845f09e2-112d-424a-88d0-e86ae1f54e9f None admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:27.204770 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845f09e2-112d-424a-88d0-e86ae1f54e9f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:57:27.205069 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845f09e2-112d-424a-88d0-e86ae1f54e9f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:27.206437 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845f09e2-112d-424a-88d0-e86ae1f54e9f None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:27.214998 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 219/437] 10.222.0.62 () {56 vars in 1111 bytes} [Wed Aug 24 05:57:27 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:57:27.221214 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b630e61d-dc0e-4bdd-9014-b40be5f4c4a7 None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:27.238724 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5903c6bb-dfd1-40b3-bbff-f39d49d595c8 None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:27.239056 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5903c6bb-dfd1-40b3-bbff-f39d49d595c8 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:27.239283 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5903c6bb-dfd1-40b3-bbff-f39d49d595c8 None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:27.240966 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 220/438] 10.222.0.62 () {54 vars in 927 bytes} [Wed Aug 24 05:57:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 05:57:27.246231 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b05272-b358-4e14-9914-dcea5f8b85d2 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:27.246706 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b05272-b358-4e14-9914-dcea5f8b85d2 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:27.247129 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b05272-b358-4e14-9914-dcea5f8b85d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:27.269044 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-87b05272-b358-4e14-9914-dcea5f8b85d2 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:57:27.270334 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-87b05272-b358-4e14-9914-dcea5f8b85d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:57:27.350401 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 221/439] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Aug 24 05:57:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 05:57:27.358099 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b05272-b358-4e14-9914-dcea5f8b85d2 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:27.361245 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b630e61d-dc0e-4bdd-9014-b40be5f4c4a7 None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:27.397782 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08136ba0-bee8-49a3-83ce-57a12db0c0df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:27.398384 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08136ba0-bee8-49a3-83ce-57a12db0c0df None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:27.398665 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08136ba0-bee8-49a3-83ce-57a12db0c0df None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:27.398940 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08136ba0-bee8-49a3-83ce-57a12db0c0df None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:27.436111 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:27.462446 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:27.466390 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b05272-b358-4e14-9914-dcea5f8b85d2 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:27.470316 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08136ba0-bee8-49a3-83ce-57a12db0c0df None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:27.471788 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08136ba0-bee8-49a3-83ce-57a12db0c0df None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:27.504777 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb2341a-53bd-47a1-a638-51dffb08169a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:27.505410 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb2341a-53bd-47a1-a638-51dffb08169a None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:27.505687 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb2341a-53bd-47a1-a638-51dffb08169a None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:27.505960 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb2341a-53bd-47a1-a638-51dffb08169a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:27.516108 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb2341a-53bd-47a1-a638-51dffb08169a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:57:27.521398 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-5eb2341a-53bd-47a1-a638-51dffb08169a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 05:57:27.521398 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:57:27.521398 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:57:27.521398 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:57:27.521398 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:57:27.521398 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:57:27.521398 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:57:27.521398 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:57:27.521398 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:57:27.521398 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:57:27.521398 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:57:27.521398 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:57:27.521398 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:57:27.521398 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:57:27.521398 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:57:27.521398 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:57:27.522269 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:57:27.522269 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:57:27.522269 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:57:27.522269 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:57:27.522269 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:57:27.522269 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:57:27.522269 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:57:27.522269 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:57:27.522269 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:57:27.522269 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:57:27.522269 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:57:27.522269 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 24 05:57:27.522269 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:57:27.522269 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:57:27.522269 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:57:27.522269 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:57:27.523109 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:57:27.523109 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:57:27.523109 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:57:27.523109 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:57:27.523109 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:57:27.523109 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:57:27.523109 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:57:27.523109 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:57:27.523109 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:57:27.523109 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:57:27.523109 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 05:57:27.523109 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 24 05:57:27.523109 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 222/440] 10.222.0.62 () {56 vars in 1114 bytes} [Wed Aug 24 05:57:27 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 05:57:27.528093 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb2341a-53bd-47a1-a638-51dffb08169a None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:27.584521 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 219/441] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Aug 24 05:57:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 05:57:27.587644 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08136ba0-bee8-49a3-83ce-57a12db0c0df None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:27.634908 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb2341a-53bd-47a1-a638-51dffb08169a None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:27.655130 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:27.667945 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae9027c-3f6a-4e11-9d15-4ddd6723ebaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:27.669944 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae9027c-3f6a-4e11-9d15-4ddd6723ebaf None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:27.669944 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae9027c-3f6a-4e11-9d15-4ddd6723ebaf None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:27.669944 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae9027c-3f6a-4e11-9d15-4ddd6723ebaf None admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:27.669944 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae9027c-3f6a-4e11-9d15-4ddd6723ebaf None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:57:27.670522 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae9027c-3f6a-4e11-9d15-4ddd6723ebaf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:27.671644 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae9027c-3f6a-4e11-9d15-4ddd6723ebaf None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:27.679315 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 223/442] 10.222.0.62 () {56 vars in 1111 bytes} [Wed Aug 24 05:57:27 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:57:27.680091 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae9027c-3f6a-4e11-9d15-4ddd6723ebaf None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:27.690778 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08136ba0-bee8-49a3-83ce-57a12db0c0df None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:27.703602 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:27.725584 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c3978e-e40a-4ec1-98d0-2222b82dd40d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:27.726199 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c3978e-e40a-4ec1-98d0-2222b82dd40d None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:27.726473 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c3978e-e40a-4ec1-98d0-2222b82dd40d None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:27.726741 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c3978e-e40a-4ec1-98d0-2222b82dd40d None admin] PATH_INFO: `/v3/projects/6d7fd5ca249a4df5aba301beb23e2eb7` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:27.727849 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c3978e-e40a-4ec1-98d0-2222b82dd40d None admin] RBAC: Authorizing `identity:get_project(project_id=6d7fd5ca249a4df5aba301beb23e2eb7)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:27.729294 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c3978e-e40a-4ec1-98d0-2222b82dd40d None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:27.730894 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 220/443] 10.222.0.62 () {56 vars in 1186 bytes} [Wed Aug 24 05:57:27 2022] GET /identity/v3/projects/6d7fd5ca249a4df5aba301beb23e2eb7 => generated 298 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:57:27.731644 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c3978e-e40a-4ec1-98d0-2222b82dd40d None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:27.787581 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae9027c-3f6a-4e11-9d15-4ddd6723ebaf None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:27.820595 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4c606f-48a7-4558-9bcf-bba2dd6b873f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:27.821216 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4c606f-48a7-4558-9bcf-bba2dd6b873f None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:27.821489 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4c606f-48a7-4558-9bcf-bba2dd6b873f None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:27.821768 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4c606f-48a7-4558-9bcf-bba2dd6b873f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:27.827978 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c3978e-e40a-4ec1-98d0-2222b82dd40d None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:27.832107 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4c606f-48a7-4558-9bcf-bba2dd6b873f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:57:27.836799 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-cc4c606f-48a7-4558-9bcf-bba2dd6b873f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 05:57:27.836799 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:57:27.836799 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:57:27.836799 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:57:27.836799 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:57:27.836799 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:57:27.836799 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:57:27.836799 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:57:27.836799 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:57:27.836799 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:57:27.836799 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:57:27.836799 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:57:27.836799 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:57:27.836799 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:57:27.836799 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:57:27.836799 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:57:27.837789 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:57:27.837789 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:57:27.837789 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:57:27.837789 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:57:27.837789 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:57:27.837789 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:57:27.837789 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:57:27.837789 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:57:27.837789 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:57:27.837789 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:57:27.837789 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:57:27.837789 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 24 05:57:27.837789 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:57:27.837789 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:57:27.837789 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:57:27.837789 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:57:27.838594 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:57:27.838594 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:57:27.838594 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:57:27.838594 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:57:27.838594 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:57:27.838594 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:57:27.838594 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:57:27.838594 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:57:27.838594 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:57:27.838594 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:57:27.838594 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 05:57:27.838594 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 24 05:57:27.838594 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 224/444] 10.222.0.62 () {56 vars in 1096 bytes} [Wed Aug 24 05:57:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:57:27.839258 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4c606f-48a7-4558-9bcf-bba2dd6b873f None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:27.863150 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9217c87-00ec-49dd-8102-0d4c07491c46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:27.863747 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9217c87-00ec-49dd-8102-0d4c07491c46 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:27.864020 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9217c87-00ec-49dd-8102-0d4c07491c46 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:27.864288 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9217c87-00ec-49dd-8102-0d4c07491c46 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:27.867600 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:27.920586 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:27.931432 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9217c87-00ec-49dd-8102-0d4c07491c46 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:27.932922 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9217c87-00ec-49dd-8102-0d4c07491c46 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:27.944056 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4c606f-48a7-4558-9bcf-bba2dd6b873f None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:27.978613 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc87d1ec-8e93-4297-9ed3-aa994fe32a0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:27.979233 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc87d1ec-8e93-4297-9ed3-aa994fe32a0c None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:27.979507 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc87d1ec-8e93-4297-9ed3-aa994fe32a0c None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:27.979773 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc87d1ec-8e93-4297-9ed3-aa994fe32a0c None admin] PATH_INFO: `/v3/groups/cec1f56499194c9fb4ba718917f95478` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:27.980905 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc87d1ec-8e93-4297-9ed3-aa994fe32a0c None admin] RBAC: Authorizing `identity:get_group(group_id=cec1f56499194c9fb4ba718917f95478)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:27.982327 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc87d1ec-8e93-4297-9ed3-aa994fe32a0c None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:27.983864 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 225/445] 10.222.0.62 () {56 vars in 1180 bytes} [Wed Aug 24 05:57:27 2022] GET /identity/v3/groups/cec1f56499194c9fb4ba718917f95478 => generated 226 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:57:27.984643 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc87d1ec-8e93-4297-9ed3-aa994fe32a0c None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:28.030644 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 221/446] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Aug 24 05:57:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 05:57:28.031477 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9217c87-00ec-49dd-8102-0d4c07491c46 None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:28.065613 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:28.082306 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc87d1ec-8e93-4297-9ed3-aa994fe32a0c None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:28.114957 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bf253c-e8ba-4080-8c14-56a0e1b8a910 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:28.115551 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bf253c-e8ba-4080-8c14-56a0e1b8a910 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:28.115825 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bf253c-e8ba-4080-8c14-56a0e1b8a910 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:28.116091 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bf253c-e8ba-4080-8c14-56a0e1b8a910 None admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:28.116765 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bf253c-e8ba-4080-8c14-56a0e1b8a910 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:57:28.117063 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bf253c-e8ba-4080-8c14-56a0e1b8a910 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:28.118410 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bf253c-e8ba-4080-8c14-56a0e1b8a910 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:28.126282 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 226/447] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Aug 24 05:57:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:57:28.127071 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bf253c-e8ba-4080-8c14-56a0e1b8a910 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:28.129834 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9217c87-00ec-49dd-8102-0d4c07491c46 None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:28.138781 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:28.163935 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913659bd-b40a-47bb-9273-9900007ad528 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:28.164551 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913659bd-b40a-47bb-9273-9900007ad528 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:28.164828 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913659bd-b40a-47bb-9273-9900007ad528 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:28.165099 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913659bd-b40a-47bb-9273-9900007ad528 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:28.165808 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913659bd-b40a-47bb-9273-9900007ad528 None admin] RBAC: Adding query filter params (group.id=cec1f56499194c9fb4ba718917f95478, role.id=ba3c2c3cb98c424f9ece19903a59c53c, scope.project.id=6d7fd5ca249a4df5aba301beb23e2eb7) {{(pid=81851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:57:28.166113 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913659bd-b40a-47bb-9273-9900007ad528 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:28.167522 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913659bd-b40a-47bb-9273-9900007ad528 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:28.173463 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 222/448] 10.222.0.62 () {56 vars in 1376 bytes} [Wed Aug 24 05:57:28 2022] GET /identity/v3/role_assignments?group.id=cec1f56499194c9fb4ba718917f95478&scope.project.id=6d7fd5ca249a4df5aba301beb23e2eb7&role.id=ba3c2c3cb98c424f9ece19903a59c53c => generated 611 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:57:28.174203 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913659bd-b40a-47bb-9273-9900007ad528 None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:28.233436 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bf253c-e8ba-4080-8c14-56a0e1b8a910 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:28.269080 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a7e2b9-398f-4059-8ab5-10943426f551 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:28.269726 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a7e2b9-398f-4059-8ab5-10943426f551 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:28.270003 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a7e2b9-398f-4059-8ab5-10943426f551 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:28.270269 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a7e2b9-398f-4059-8ab5-10943426f551 None admin] PATH_INFO: `/v3/projects/9391bbe626b44799942ef25c41bc33ad` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:28.271432 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a7e2b9-398f-4059-8ab5-10943426f551 None admin] RBAC: Authorizing `identity:get_project(project_id=9391bbe626b44799942ef25c41bc33ad)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:28.272922 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a7e2b9-398f-4059-8ab5-10943426f551 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:28.274601 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 227/449] 10.222.0.62 () {56 vars in 1186 bytes} [Wed Aug 24 05:57:28 2022] GET /identity/v3/projects/9391bbe626b44799942ef25c41bc33ad => generated 302 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:57:28.280269 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a7e2b9-398f-4059-8ab5-10943426f551 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:28.287557 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:28.290138 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913659bd-b40a-47bb-9273-9900007ad528 None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:28.329667 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6bc92a-abde-494e-9eb8-672b2549d8ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:28.330297 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6bc92a-abde-494e-9eb8-672b2549d8ee None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:28.330563 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6bc92a-abde-494e-9eb8-672b2549d8ee None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:28.330838 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6bc92a-abde-494e-9eb8-672b2549d8ee None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:28.374377 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:28.397619 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a7e2b9-398f-4059-8ab5-10943426f551 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:28.417914 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6bc92a-abde-494e-9eb8-672b2549d8ee None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:28.419385 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6bc92a-abde-494e-9eb8-672b2549d8ee None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:28.434113 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a00ce33-ba83-4500-8bde-f6ece0bda3c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:28.434745 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a00ce33-ba83-4500-8bde-f6ece0bda3c9 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:28.435021 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a00ce33-ba83-4500-8bde-f6ece0bda3c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:28.435356 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a00ce33-ba83-4500-8bde-f6ece0bda3c9 None admin] PATH_INFO: `/v3/groups/cec1f56499194c9fb4ba718917f95478` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:28.436558 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a00ce33-ba83-4500-8bde-f6ece0bda3c9 None admin] RBAC: Authorizing `identity:get_group(group_id=cec1f56499194c9fb4ba718917f95478)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:28.437952 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a00ce33-ba83-4500-8bde-f6ece0bda3c9 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:28.440013 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 228/450] 10.222.0.62 () {56 vars in 1180 bytes} [Wed Aug 24 05:57:28 2022] GET /identity/v3/groups/cec1f56499194c9fb4ba718917f95478 => generated 226 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:57:28.445922 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a00ce33-ba83-4500-8bde-f6ece0bda3c9 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:28.540106 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:28.544210 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 223/451] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Aug 24 05:57:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 05:57:28.551058 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6bc92a-abde-494e-9eb8-672b2549d8ee None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:28.566863 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a00ce33-ba83-4500-8bde-f6ece0bda3c9 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:28.605653 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71f82e0-bc49-48ce-bfbf-e57692faac79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:28.606546 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71f82e0-bc49-48ce-bfbf-e57692faac79 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:28.606993 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71f82e0-bc49-48ce-bfbf-e57692faac79 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:28.607458 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71f82e0-bc49-48ce-bfbf-e57692faac79 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:28.608342 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71f82e0-bc49-48ce-bfbf-e57692faac79 None admin] RBAC: Adding query filter params (group.id=cec1f56499194c9fb4ba718917f95478, role.id=ba3c2c3cb98c424f9ece19903a59c53c, scope.project.id=9391bbe626b44799942ef25c41bc33ad) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:57:28.608844 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71f82e0-bc49-48ce-bfbf-e57692faac79 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:28.610499 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71f82e0-bc49-48ce-bfbf-e57692faac79 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:28.616431 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 229/452] 10.222.0.62 () {56 vars in 1376 bytes} [Wed Aug 24 05:57:28 2022] GET /identity/v3/role_assignments?group.id=cec1f56499194c9fb4ba718917f95478&scope.project.id=9391bbe626b44799942ef25c41bc33ad&role.id=ba3c2c3cb98c424f9ece19903a59c53c => generated 611 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:57:28.676635 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6bc92a-abde-494e-9eb8-672b2549d8ee None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:28.722924 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a87d964-7185-4b83-b9cc-c7e03b14a5c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:28.723573 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a87d964-7185-4b83-b9cc-c7e03b14a5c0 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:28.723855 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a87d964-7185-4b83-b9cc-c7e03b14a5c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:28.724129 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a87d964-7185-4b83-b9cc-c7e03b14a5c0 None admin] PATH_INFO: `/v3/projects/632e93ce840f4853b64b4d659536e045` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:28.725341 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a87d964-7185-4b83-b9cc-c7e03b14a5c0 None admin] RBAC: Authorizing `identity:get_project(project_id=632e93ce840f4853b64b4d659536e045)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:28.726824 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a87d964-7185-4b83-b9cc-c7e03b14a5c0 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:28.728545 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 224/453] 10.222.0.62 () {56 vars in 1186 bytes} [Wed Aug 24 05:57:28 2022] GET /identity/v3/projects/632e93ce840f4853b64b4d659536e045 => generated 344 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:57:28.734259 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71f82e0-bc49-48ce-bfbf-e57692faac79 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:28.747803 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:28.837046 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71f82e0-bc49-48ce-bfbf-e57692faac79 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:28.870262 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c75e820-1bdf-4335-b690-cb9f30e1922b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:28.870896 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c75e820-1bdf-4335-b690-cb9f30e1922b None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:28.871178 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c75e820-1bdf-4335-b690-cb9f30e1922b None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:28.871453 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c75e820-1bdf-4335-b690-cb9f30e1922b None admin] PATH_INFO: `/v3/groups/cbf8f7f1550c4404831928e6ad3a3393` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:28.872619 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c75e820-1bdf-4335-b690-cb9f30e1922b None admin] RBAC: Authorizing `identity:get_group(group_id=cbf8f7f1550c4404831928e6ad3a3393)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:28.874062 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c75e820-1bdf-4335-b690-cb9f30e1922b None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:28.875675 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 230/454] 10.222.0.62 () {56 vars in 1180 bytes} [Wed Aug 24 05:57:28 2022] GET /identity/v3/groups/cbf8f7f1550c4404831928e6ad3a3393 => generated 229 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:57:28.882269 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a87d964-7185-4b83-b9cc-c7e03b14a5c0 None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:28.951843 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:28.985675 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a87d964-7185-4b83-b9cc-c7e03b14a5c0 None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:29.018096 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d9ddc2-d338-448a-b3d3-a03937601bc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:29.018685 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d9ddc2-d338-448a-b3d3-a03937601bc7 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:29.018963 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d9ddc2-d338-448a-b3d3-a03937601bc7 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:29.019235 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d9ddc2-d338-448a-b3d3-a03937601bc7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:29.019946 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d9ddc2-d338-448a-b3d3-a03937601bc7 None admin] RBAC: Adding query filter params (group.id=cbf8f7f1550c4404831928e6ad3a3393, role.id=49e47cd157d743f6905f1db07ec878a7, scope.project.id=632e93ce840f4853b64b4d659536e045) {{(pid=81851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:57:29.020254 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d9ddc2-d338-448a-b3d3-a03937601bc7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:29.021723 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d9ddc2-d338-448a-b3d3-a03937601bc7 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:29.027281 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 225/455] 10.222.0.62 () {56 vars in 1376 bytes} [Wed Aug 24 05:57:28 2022] GET /identity/v3/role_assignments?group.id=cbf8f7f1550c4404831928e6ad3a3393&scope.project.id=632e93ce840f4853b64b4d659536e045&role.id=49e47cd157d743f6905f1db07ec878a7 => generated 611 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:57:31.390124 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44696bf4-a2d9-4b16-8588-29a2f530456f None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:31.390661 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44696bf4-a2d9-4b16-8588-29a2f530456f None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:31.390661 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44696bf4-a2d9-4b16-8588-29a2f530456f None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:31.392347 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 231/456] 10.222.0.62 () {54 vars in 927 bytes} [Wed Aug 24 05:57:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:57:31.397940 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ce8428-648e-4fa5-b163-aaf3b2da6d76 None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:31.398780 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ce8428-648e-4fa5-b163-aaf3b2da6d76 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:31.399258 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ce8428-648e-4fa5-b163-aaf3b2da6d76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:31.429448 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-62ce8428-648e-4fa5-b163-aaf3b2da6d76 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:57:31.430990 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-62ce8428-648e-4fa5-b163-aaf3b2da6d76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:57:31.518700 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 226/457] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Aug 24 05:57:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 05:57:31.558768 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1e216e-1d64-4d6b-aa74-e72f37b3932c None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:31.559385 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1e216e-1d64-4d6b-aa74-e72f37b3932c None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:31.559863 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1e216e-1d64-4d6b-aa74-e72f37b3932c None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:31.561437 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 232/458] 10.222.0.62 () {54 vars in 927 bytes} [Wed Aug 24 05:57:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:57:31.566919 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e1a1eb-997d-43f3-a8ae-a37b90464d63 None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:31.567462 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e1a1eb-997d-43f3-a8ae-a37b90464d63 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:31.567933 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e1a1eb-997d-43f3-a8ae-a37b90464d63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:31.597059 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-96e1a1eb-997d-43f3-a8ae-a37b90464d63 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:57:31.598610 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-96e1a1eb-997d-43f3-a8ae-a37b90464d63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:57:31.636760 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55db956d-65c0-44cb-ba6e-5c2fbf92b380 None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:31.637432 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55db956d-65c0-44cb-ba6e-5c2fbf92b380 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:31.637783 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55db956d-65c0-44cb-ba6e-5c2fbf92b380 None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:31.639587 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 233/459] 10.222.0.62 () {54 vars in 927 bytes} [Wed Aug 24 05:57:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:57:31.650779 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4125b79a-31d0-403b-a22f-7ba5136dea49 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:31.651418 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4125b79a-31d0-403b-a22f-7ba5136dea49 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:31.652095 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4125b79a-31d0-403b-a22f-7ba5136dea49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:31.680882 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:31.689483 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-4125b79a-31d0-403b-a22f-7ba5136dea49 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:57:31.690894 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-4125b79a-31d0-403b-a22f-7ba5136dea49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:57:31.691013 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 227/460] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Aug 24 05:57:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 05:57:31.803059 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 234/461] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Aug 24 05:57:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 05:57:31.835385 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e1a1eb-997d-43f3-a8ae-a37b90464d63 None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:31.952025 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e1a1eb-997d-43f3-a8ae-a37b90464d63 None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:31.992294 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3deefc52-79f8-4b6c-ba06-f75b925f8bcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:31.992978 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3deefc52-79f8-4b6c-ba06-f75b925f8bcc None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:31.993255 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3deefc52-79f8-4b6c-ba06-f75b925f8bcc None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:31.993525 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3deefc52-79f8-4b6c-ba06-f75b925f8bcc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:32.009321 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3deefc52-79f8-4b6c-ba06-f75b925f8bcc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:32.010792 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3deefc52-79f8-4b6c-ba06-f75b925f8bcc None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:32.025455 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application [None req-3deefc52-79f8-4b6c-ba06-f75b925f8bcc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:57:32.025455 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:57:32.025455 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:57:32.025455 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:57:32.025455 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:57:32.025455 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:57:32.025455 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:57:32.025455 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:57:32.025455 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:57:32.025455 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:57:32.025455 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:57:32.025455 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:57:32.025455 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 05:57:32.025455 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 05:57:32.025455 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 05:57:32.025455 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 05:57:32.026381 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:57:32.026381 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:57:32.026381 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:57:32.026381 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:57:32.026381 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:57:32.026381 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:57:32.026381 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:57:32.026381 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application with Lock( Aug 24 05:57:32.026381 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:57:32.026381 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:57:32.026381 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:57:32.026381 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:57:32.026381 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:57:32.026381 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:57:32.026381 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:57:32.026381 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:57:32.026381 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 05:57:32.027239 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 05:57:32.027239 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:57:32.027239 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Aug 24 05:57:32.027239 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 228/462] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Aug 24 05:57:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:57:32.032515 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4125b79a-31d0-403b-a22f-7ba5136dea49 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:32.050361 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:32.078849 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3deefc52-79f8-4b6c-ba06-f75b925f8bcc None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:32.126696 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:32.173077 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4125b79a-31d0-403b-a22f-7ba5136dea49 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:32.188314 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3deefc52-79f8-4b6c-ba06-f75b925f8bcc None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:32.209446 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb94b1d-85d5-48cd-8806-21d985eb51de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:32.210011 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb94b1d-85d5-48cd-8806-21d985eb51de None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:32.210291 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb94b1d-85d5-48cd-8806-21d985eb51de None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:32.210581 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb94b1d-85d5-48cd-8806-21d985eb51de None admin] PATH_INFO: `/v3/domains` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:32.211204 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb94b1d-85d5-48cd-8806-21d985eb51de None admin] RBAC: Adding query filter params (name=Default) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:57:32.211509 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb94b1d-85d5-48cd-8806-21d985eb51de None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:32.213047 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb94b1d-85d5-48cd-8806-21d985eb51de None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:32.225038 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038d0214-8b25-4908-8e36-11e9abd73991 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:32.225843 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 235/463] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Aug 24 05:57:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:57:32.226180 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038d0214-8b25-4908-8e36-11e9abd73991 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:32.226716 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb94b1d-85d5-48cd-8806-21d985eb51de None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:32.226869 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038d0214-8b25-4908-8e36-11e9abd73991 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:32.227166 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038d0214-8b25-4908-8e36-11e9abd73991 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:32.240790 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038d0214-8b25-4908-8e36-11e9abd73991 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:32.242557 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038d0214-8b25-4908-8e36-11e9abd73991 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:32.257652 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application [None req-038d0214-8b25-4908-8e36-11e9abd73991 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:57:32.257652 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:57:32.257652 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:57:32.257652 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:57:32.257652 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:57:32.257652 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:57:32.257652 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:57:32.257652 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:57:32.257652 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:57:32.257652 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:57:32.257652 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:57:32.257652 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:57:32.257652 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 05:57:32.257652 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 05:57:32.257652 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 05:57:32.257652 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 05:57:32.258589 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:57:32.258589 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:57:32.258589 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:57:32.258589 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:57:32.258589 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:57:32.258589 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:57:32.258589 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:57:32.258589 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application with Lock( Aug 24 05:57:32.258589 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:57:32.258589 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:57:32.258589 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:57:32.258589 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:57:32.258589 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:57:32.258589 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:57:32.258589 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:57:32.258589 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:57:32.258589 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 05:57:32.259422 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 05:57:32.259422 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:57:32.259422 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Aug 24 05:57:32.264625 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 229/464] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Aug 24 05:57:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 05:57:32.265678 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038d0214-8b25-4908-8e36-11e9abd73991 None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:32.297094 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:32.332844 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb94b1d-85d5-48cd-8806-21d985eb51de None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:32.366713 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:32.367645 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043be66d-0d9e-42e6-8c30-d4ca241d00dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:32.368267 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043be66d-0d9e-42e6-8c30-d4ca241d00dd None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:32.368577 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043be66d-0d9e-42e6-8c30-d4ca241d00dd None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:32.368858 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043be66d-0d9e-42e6-8c30-d4ca241d00dd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:32.372688 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038d0214-8b25-4908-8e36-11e9abd73991 None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:32.381091 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043be66d-0d9e-42e6-8c30-d4ca241d00dd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:32.382618 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043be66d-0d9e-42e6-8c30-d4ca241d00dd None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:32.396923 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-043be66d-0d9e-42e6-8c30-d4ca241d00dd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:57:32.396923 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:57:32.396923 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:57:32.396923 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:57:32.396923 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:57:32.396923 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:57:32.396923 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:57:32.396923 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:57:32.396923 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:57:32.396923 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:57:32.396923 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:57:32.396923 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:57:32.396923 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 05:57:32.396923 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 05:57:32.396923 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 05:57:32.396923 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 05:57:32.401267 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:57:32.401267 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:57:32.401267 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:57:32.401267 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:57:32.401267 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:57:32.401267 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:57:32.401267 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:57:32.401267 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 24 05:57:32.401267 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:57:32.401267 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:57:32.401267 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:57:32.401267 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:57:32.401267 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:57:32.401267 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:57:32.401267 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:57:32.401267 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:57:32.401267 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 05:57:32.402106 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 05:57:32.402106 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:57:32.402106 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 24 05:57:32.402106 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 236/465] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Aug 24 05:57:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:57:32.402106 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043be66d-0d9e-42e6-8c30-d4ca241d00dd None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:32.414099 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069d141d-809f-450d-ab77-0d57fc121bca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:32.414767 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069d141d-809f-450d-ab77-0d57fc121bca None admin] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:32.415033 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069d141d-809f-450d-ab77-0d57fc121bca None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:32.415279 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069d141d-809f-450d-ab77-0d57fc121bca None admin] PATH_INFO: `/v3/users` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:32.415917 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069d141d-809f-450d-ab77-0d57fc121bca None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:32.417392 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069d141d-809f-450d-ab77-0d57fc121bca None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:32.502264 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043be66d-0d9e-42e6-8c30-d4ca241d00dd None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:32.523349 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:32.525375 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 230/466] 10.222.0.62 () {60 vars in 1132 bytes} [Wed Aug 24 05:57:32 2022] POST /identity/v3/users => generated 246 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:57:32.526788 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec74ed7-4e7f-46fa-9cec-50e411d2e219 None None] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:32.527092 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec74ed7-4e7f-46fa-9cec-50e411d2e219 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:32.527359 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec74ed7-4e7f-46fa-9cec-50e411d2e219 None None] PATH_INFO: `/` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:32.528599 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 231/467] 10.222.0.62 () {54 vars in 927 bytes} [Wed Aug 24 05:57:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:57:32.529673 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec74ed7-4e7f-46fa-9cec-50e411d2e219 None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:32.551652 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa10ed0-be85-486f-819b-ea804fd4ad65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:32.552370 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa10ed0-be85-486f-819b-ea804fd4ad65 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:32.552691 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa10ed0-be85-486f-819b-ea804fd4ad65 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:32.553036 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa10ed0-be85-486f-819b-ea804fd4ad65 None admin] PATH_INFO: `/v3/domains` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:32.553702 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa10ed0-be85-486f-819b-ea804fd4ad65 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:57:32.554009 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa10ed0-be85-486f-819b-ea804fd4ad65 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:32.555483 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa10ed0-be85-486f-819b-ea804fd4ad65 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:32.569163 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 237/468] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Aug 24 05:57:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:57:32.573963 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3681d6-3ce9-4819-a2bb-527bb919fd7e None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:32.573963 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3681d6-3ce9-4819-a2bb-527bb919fd7e None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:32.573963 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3681d6-3ce9-4819-a2bb-527bb919fd7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:32.590961 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-1b3681d6-3ce9-4819-a2bb-527bb919fd7e None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:57:32.592204 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-1b3681d6-3ce9-4819-a2bb-527bb919fd7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:57:32.676939 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 238/469] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Aug 24 05:57:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 05:57:32.677984 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3681d6-3ce9-4819-a2bb-527bb919fd7e None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:32.679567 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec74ed7-4e7f-46fa-9cec-50e411d2e219 None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:32.731192 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84274a68-c008-472c-a9f5-82f06f590d02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:32.732119 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84274a68-c008-472c-a9f5-82f06f590d02 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:32.732657 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84274a68-c008-472c-a9f5-82f06f590d02 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:32.733139 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84274a68-c008-472c-a9f5-82f06f590d02 None admin] PATH_INFO: `/v3/domains` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:32.733973 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84274a68-c008-472c-a9f5-82f06f590d02 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:57:32.734534 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84274a68-c008-472c-a9f5-82f06f590d02 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:32.736114 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84274a68-c008-472c-a9f5-82f06f590d02 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:32.763858 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 232/470] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Aug 24 05:57:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:57:32.767426 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84274a68-c008-472c-a9f5-82f06f590d02 None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:32.784869 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:32.811019 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3681d6-3ce9-4819-a2bb-527bb919fd7e None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:32.822654 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:32.849959 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1664f0a-e7d7-4cb2-b1a9-8a978fbebb5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:32.849959 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1664f0a-e7d7-4cb2-b1a9-8a978fbebb5b None admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:32.850366 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1664f0a-e7d7-4cb2-b1a9-8a978fbebb5b None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:32.850545 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1664f0a-e7d7-4cb2-b1a9-8a978fbebb5b None admin] PATH_INFO: `/v3/users` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:32.851170 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1664f0a-e7d7-4cb2-b1a9-8a978fbebb5b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:32.852620 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1664f0a-e7d7-4cb2-b1a9-8a978fbebb5b None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:32.882505 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84274a68-c008-472c-a9f5-82f06f590d02 None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:32.919655 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd4a1c3-c141-4fcb-a60c-4c38e09df777 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:32.920534 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd4a1c3-c141-4fcb-a60c-4c38e09df777 None admin] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:32.920816 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd4a1c3-c141-4fcb-a60c-4c38e09df777 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:32.921083 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd4a1c3-c141-4fcb-a60c-4c38e09df777 None admin] PATH_INFO: `/v3/users` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:32.921710 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd4a1c3-c141-4fcb-a60c-4c38e09df777 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:32.923191 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd4a1c3-c141-4fcb-a60c-4c38e09df777 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:32.971320 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 239/471] 10.222.0.62 () {60 vars in 1132 bytes} [Wed Aug 24 05:57:32 2022] POST /identity/v3/users => generated 248 bytes in 294 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:57:32.972574 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994de88b-7407-4d54-9200-405c38af0606 None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:32.972877 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994de88b-7407-4d54-9200-405c38af0606 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:32.973158 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994de88b-7407-4d54-9200-405c38af0606 None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:32.974473 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 240/472] 10.222.0.62 () {54 vars in 927 bytes} [Wed Aug 24 05:57:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:57:32.976705 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994de88b-7407-4d54-9200-405c38af0606 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:33.036619 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 233/473] 10.222.0.62 () {60 vars in 1132 bytes} [Wed Aug 24 05:57:32 2022] POST /identity/v3/users => generated 248 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:57:33.039697 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af704de-f9ed-4646-a1df-0e951e95229c None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:33.040194 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af704de-f9ed-4646-a1df-0e951e95229c None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:33.040649 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af704de-f9ed-4646-a1df-0e951e95229c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:33.084788 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-8af704de-f9ed-4646-a1df-0e951e95229c None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:57:33.087977 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-8af704de-f9ed-4646-a1df-0e951e95229c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:57:33.143221 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994de88b-7407-4d54-9200-405c38af0606 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:33.184376 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b11e04-1f0e-43e6-a040-6b6ff51d3c23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:33.185353 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b11e04-1f0e-43e6-a040-6b6ff51d3c23 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:33.185464 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:33.185632 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b11e04-1f0e-43e6-a040-6b6ff51d3c23 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:33.186114 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b11e04-1f0e-43e6-a040-6b6ff51d3c23 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:33.193006 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 234/474] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Aug 24 05:57:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 05:57:33.199661 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b11e04-1f0e-43e6-a040-6b6ff51d3c23 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:33.201331 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b11e04-1f0e-43e6-a040-6b6ff51d3c23 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:33.216984 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-b2b11e04-1f0e-43e6-a040-6b6ff51d3c23 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:57:33.216984 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:57:33.216984 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:57:33.216984 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:57:33.216984 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:57:33.216984 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:57:33.216984 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:57:33.216984 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:57:33.216984 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:57:33.216984 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:57:33.216984 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:57:33.216984 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:57:33.216984 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 05:57:33.216984 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 05:57:33.216984 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 05:57:33.216984 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 05:57:33.217946 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:57:33.217946 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:57:33.217946 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:57:33.217946 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:57:33.217946 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:57:33.217946 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:57:33.217946 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:57:33.217946 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 24 05:57:33.217946 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:57:33.217946 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:57:33.217946 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:57:33.217946 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:57:33.217946 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:57:33.217946 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:57:33.217946 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:57:33.217946 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:57:33.217946 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 05:57:33.219626 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 05:57:33.219626 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:57:33.219626 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 24 05:57:33.219626 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 241/475] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Aug 24 05:57:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:57:33.224838 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af704de-f9ed-4646-a1df-0e951e95229c None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:33.332829 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af704de-f9ed-4646-a1df-0e951e95229c None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:33.368835 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b586983f-3302-4d31-9551-a060a6a3c545 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:33.369501 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b586983f-3302-4d31-9551-a060a6a3c545 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:33.369775 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b586983f-3302-4d31-9551-a060a6a3c545 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:33.370042 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b586983f-3302-4d31-9551-a060a6a3c545 None admin] PATH_INFO: `/v3/domains` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:33.370715 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b586983f-3302-4d31-9551-a060a6a3c545 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:57:33.371012 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b586983f-3302-4d31-9551-a060a6a3c545 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:33.372379 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b586983f-3302-4d31-9551-a060a6a3c545 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:33.384668 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 235/476] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Aug 24 05:57:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:57:33.390211 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b11e04-1f0e-43e6-a040-6b6ff51d3c23 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:33.413108 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:33.501169 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b586983f-3302-4d31-9551-a060a6a3c545 None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:33.504437 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b11e04-1f0e-43e6-a040-6b6ff51d3c23 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:33.554053 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd51328-6137-4987-862d-c5aad8e41294 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:33.554655 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd51328-6137-4987-862d-c5aad8e41294 None admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:33.555178 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd51328-6137-4987-862d-c5aad8e41294 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:33.555454 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd51328-6137-4987-862d-c5aad8e41294 None admin] PATH_INFO: `/v3/users` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:33.557158 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd51328-6137-4987-862d-c5aad8e41294 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:33.560793 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd51328-6137-4987-862d-c5aad8e41294 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:33.583072 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:33.626134 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b586983f-3302-4d31-9551-a060a6a3c545 None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:33.661382 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0177767-0bce-472c-aa4d-a7bfdef34404 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:33.662422 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0177767-0bce-472c-aa4d-a7bfdef34404 None admin] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:33.662925 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0177767-0bce-472c-aa4d-a7bfdef34404 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:33.663414 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0177767-0bce-472c-aa4d-a7bfdef34404 None admin] PATH_INFO: `/v3/roles` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:33.664283 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0177767-0bce-472c-aa4d-a7bfdef34404 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:33.665871 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0177767-0bce-472c-aa4d-a7bfdef34404 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:33.698654 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 242/477] 10.222.0.62 () {60 vars in 1132 bytes} [Wed Aug 24 05:57:33 2022] POST /identity/v3/users => generated 249 bytes in 309 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:57:33.708105 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.sql.core [None req-d0177767-0bce-472c-aa4d-a7bfdef34404 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 24 05:57:33.708105 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 24 05:57:33.708105 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [parameters: {'id': '80d463058b6f4074b3aaab11b02f4a94', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 24 05:57:33.708105 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81851) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 24 05:57:33.733102 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application [None req-d0177767-0bce-472c-aa4d-a7bfdef34404 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 24 05:57:33.733102 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:57:33.733102 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 24 05:57:33.733102 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 24 05:57:33.733102 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 24 05:57:33.733102 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 24 05:57:33.733102 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 24 05:57:33.733102 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application result = self._query(query) Aug 24 05:57:33.733102 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 24 05:57:33.733102 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application conn.query(q) Aug 24 05:57:33.733102 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 24 05:57:33.733102 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 24 05:57:33.733102 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 24 05:57:33.733102 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application result.read() Aug 24 05:57:33.733102 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 24 05:57:33.734050 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 24 05:57:33.734050 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 24 05:57:33.734050 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 24 05:57:33.734050 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 24 05:57:33.734050 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 24 05:57:33.734050 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 24 05:57:33.734050 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 24 05:57:33.734050 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 24 05:57:33.734050 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Aug 24 05:57:33.734050 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 24 05:57:33.734050 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Aug 24 05:57:33.734050 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:57:33.734050 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 24 05:57:33.734050 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 24 05:57:33.734050 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Aug 24 05:57:33.734050 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return ref.to_dict() Aug 24 05:57:33.734050 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 24 05:57:33.734050 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application next(self.gen) Aug 24 05:57:33.735001 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 24 05:57:33.735001 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application yield resource Aug 24 05:57:33.735001 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 24 05:57:33.735001 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application next(self.gen) Aug 24 05:57:33.735001 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 24 05:57:33.735001 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application self.session.rollback() Aug 24 05:57:33.735001 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 24 05:57:33.735001 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application self.force_reraise() Aug 24 05:57:33.735001 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 24 05:57:33.735001 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application raise self.value Aug 24 05:57:33.735001 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 24 05:57:33.735001 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 24 05:57:33.735001 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 24 05:57:33.735001 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application session.commit() Aug 24 05:57:33.735001 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 24 05:57:33.735001 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 24 05:57:33.735001 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 24 05:57:33.735861 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application self._prepare_impl() Aug 24 05:57:33.735861 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 24 05:57:33.735861 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application self.session.flush() Aug 24 05:57:33.735861 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 24 05:57:33.735861 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application self._flush(objects) Aug 24 05:57:33.735861 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 24 05:57:33.735861 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 24 05:57:33.735861 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 24 05:57:33.735861 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application compat.raise_( Aug 24 05:57:33.735861 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 24 05:57:33.735861 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application raise exception Aug 24 05:57:33.735861 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 24 05:57:33.735861 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application flush_context.execute() Aug 24 05:57:33.735861 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 24 05:57:33.735861 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application rec.execute(self) Aug 24 05:57:33.735861 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 24 05:57:33.735861 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 24 05:57:33.737615 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 24 05:57:33.737615 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 24 05:57:33.737615 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 24 05:57:33.737615 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 24 05:57:33.737615 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 24 05:57:33.737615 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 24 05:57:33.737615 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 24 05:57:33.737615 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 24 05:57:33.737615 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 24 05:57:33.737615 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 24 05:57:33.737615 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 24 05:57:33.737615 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 24 05:57:33.737615 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 24 05:57:33.737615 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 24 05:57:33.737615 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 24 05:57:33.737615 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application raise exception Aug 24 05:57:33.739221 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 24 05:57:33.739221 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 24 05:57:33.739221 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 24 05:57:33.739221 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 24 05:57:33.739221 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 24 05:57:33.739221 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application result = self._query(query) Aug 24 05:57:33.739221 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 24 05:57:33.739221 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application conn.query(q) Aug 24 05:57:33.739221 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 24 05:57:33.739221 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 24 05:57:33.739221 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 24 05:57:33.739221 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application result.read() Aug 24 05:57:33.739221 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 24 05:57:33.739221 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 24 05:57:33.739221 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 24 05:57:33.739221 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 24 05:57:33.741179 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 24 05:57:33.741179 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 24 05:57:33.741179 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 24 05:57:33.741179 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 24 05:57:33.741179 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 24 05:57:33.741179 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 24 05:57:33.741179 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application [parameters: {'id': '80d463058b6f4074b3aaab11b02f4a94', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 24 05:57:33.741179 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 24 05:57:33.741179 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Aug 24 05:57:33.741179 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 05:57:33.741179 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Aug 24 05:57:33.741179 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:57:33.741179 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:57:33.741179 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:57:33.741179 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:57:33.741179 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:57:33.743169 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:57:33.743169 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:57:33.743169 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:57:33.743169 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:57:33.743169 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:57:33.743169 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:57:33.743169 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Aug 24 05:57:33.743169 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Aug 24 05:57:33.743169 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:57:33.743169 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:57:33.743169 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Aug 24 05:57:33.743169 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Aug 24 05:57:33.743169 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 24 05:57:33.743169 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 24 05:57:33.743169 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 24 05:57:33.743169 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Aug 24 05:57:33.749340 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 236/478] 10.222.0.62 () {60 vars in 1131 bytes} [Wed Aug 24 05:57:33 2022] POST /identity/v3/roles => generated 144 bytes in 234 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Aug 24 05:57:33.752523 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd51328-6137-4987-862d-c5aad8e41294 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:33.878811 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:33.912658 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd51328-6137-4987-862d-c5aad8e41294 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:33.963106 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56da2e1-ccb2-4f5f-8fe5-0aff92eed8c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:33.964210 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56da2e1-ccb2-4f5f-8fe5-0aff92eed8c8 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:33.964712 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56da2e1-ccb2-4f5f-8fe5-0aff92eed8c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:33.965175 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56da2e1-ccb2-4f5f-8fe5-0aff92eed8c8 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:33.980399 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56da2e1-ccb2-4f5f-8fe5-0aff92eed8c8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:57:33.986179 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-b56da2e1-ccb2-4f5f-8fe5-0aff92eed8c8 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 05:57:33.986179 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:57:33.986179 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:57:33.986179 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:57:33.986179 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:57:33.986179 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:57:33.986179 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:57:33.986179 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:57:33.986179 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:57:33.986179 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:57:33.986179 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:57:33.986179 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:57:33.986179 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:57:33.986179 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:57:33.986179 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:57:33.986179 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:57:33.987069 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:57:33.987069 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:57:33.987069 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:57:33.987069 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:57:33.987069 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:57:33.987069 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:57:33.987069 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:57:33.987069 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:57:33.987069 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:57:33.987069 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:57:33.987069 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:57:33.987069 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 24 05:57:33.987069 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:57:33.987069 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:57:33.987069 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:57:33.987069 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:57:33.987855 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:57:33.987855 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:57:33.987855 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:57:33.987855 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:57:33.987855 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:57:33.987855 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:57:33.987855 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:57:33.987855 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:57:33.987855 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:57:33.987855 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:57:33.987855 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 05:57:33.987855 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 24 05:57:33.990098 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 243/479] 10.222.0.62 () {56 vars in 1114 bytes} [Wed Aug 24 05:57:33 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:57:33.995909 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0177767-0bce-472c-aa4d-a7bfdef34404 None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:34.100897 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0177767-0bce-472c-aa4d-a7bfdef34404 None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:34.124119 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:34.136808 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b189a416-8d70-4677-81f3-cea0b64432d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:34.137509 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b189a416-8d70-4677-81f3-cea0b64432d3 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:34.137888 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b189a416-8d70-4677-81f3-cea0b64432d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:34.138251 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b189a416-8d70-4677-81f3-cea0b64432d3 None admin] PATH_INFO: `/v3/roles` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:34.139016 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b189a416-8d70-4677-81f3-cea0b64432d3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:57:34.139408 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b189a416-8d70-4677-81f3-cea0b64432d3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:34.140922 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b189a416-8d70-4677-81f3-cea0b64432d3 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:34.149964 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 237/480] 10.222.0.62 () {56 vars in 1111 bytes} [Wed Aug 24 05:57:33 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:57:34.811969 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b4a6d0-8ece-4b99-ab67-a5af7213533e None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:34.812529 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b4a6d0-8ece-4b99-ab67-a5af7213533e None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:34.812529 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b4a6d0-8ece-4b99-ab67-a5af7213533e None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:34.814678 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 244/481] 10.222.0.62 () {54 vars in 927 bytes} [Wed Aug 24 05:57:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:57:34.819585 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422a26a9-8e3d-43b4-9bb9-fddf2696c218 None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:34.819988 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422a26a9-8e3d-43b4-9bb9-fddf2696c218 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:34.826376 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422a26a9-8e3d-43b4-9bb9-fddf2696c218 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:34.845439 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-422a26a9-8e3d-43b4-9bb9-fddf2696c218 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:57:34.846619 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-422a26a9-8e3d-43b4-9bb9-fddf2696c218 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:57:34.927016 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 238/482] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Aug 24 05:57:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 05:57:35.318681 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b4a6d0-8ece-4b99-ab67-a5af7213533e None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:35.352905 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3112f18-4729-44de-b282-583b263c8471 None None] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:35.353551 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3112f18-4729-44de-b282-583b263c8471 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:35.354036 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3112f18-4729-44de-b282-583b263c8471 None None] PATH_INFO: `/` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:35.356070 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 239/483] 10.222.0.62 () {54 vars in 927 bytes} [Wed Aug 24 05:57:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:57:35.362295 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c0587f-9b1f-4f27-8159-0c99888e56e3 None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:35.362809 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c0587f-9b1f-4f27-8159-0c99888e56e3 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:35.363306 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c0587f-9b1f-4f27-8159-0c99888e56e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:35.381821 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-75c0587f-9b1f-4f27-8159-0c99888e56e3 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:57:35.383430 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-75c0587f-9b1f-4f27-8159-0c99888e56e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:57:35.460462 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b4a6d0-8ece-4b99-ab67-a5af7213533e None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:35.492104 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 240/484] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Aug 24 05:57:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 05:57:35.492104 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28253be3-f2ca-446c-acdb-5fccd122d510 None None] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:35.492104 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28253be3-f2ca-446c-acdb-5fccd122d510 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:35.492104 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28253be3-f2ca-446c-acdb-5fccd122d510 None None] PATH_INFO: `/` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:35.492104 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 241/485] 10.222.0.62 () {54 vars in 927 bytes} [Wed Aug 24 05:57:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:57:35.497503 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f463b3b-92b8-40a9-946e-49b9f0587323 None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:35.497791 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f463b3b-92b8-40a9-946e-49b9f0587323 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:35.498072 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f463b3b-92b8-40a9-946e-49b9f0587323 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:35.511323 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68041c4-ba24-4c4e-96d0-8c04c29cef76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:35.512457 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68041c4-ba24-4c4e-96d0-8c04c29cef76 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:35.512986 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68041c4-ba24-4c4e-96d0-8c04c29cef76 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:35.513505 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68041c4-ba24-4c4e-96d0-8c04c29cef76 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:35.515682 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:35.522865 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-0f463b3b-92b8-40a9-946e-49b9f0587323 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:57:35.524100 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-0f463b3b-92b8-40a9-946e-49b9f0587323 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:57:35.548941 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68041c4-ba24-4c4e-96d0-8c04c29cef76 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:57:35.555364 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-f68041c4-ba24-4c4e-96d0-8c04c29cef76 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 05:57:35.555364 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:57:35.555364 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:57:35.555364 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:57:35.555364 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:57:35.555364 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:57:35.555364 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:57:35.555364 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:57:35.555364 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:57:35.555364 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:57:35.555364 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:57:35.555364 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:57:35.555364 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:57:35.555364 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:57:35.555364 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:57:35.555364 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:57:35.556235 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:57:35.556235 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:57:35.556235 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:57:35.556235 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:57:35.556235 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:57:35.556235 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:57:35.556235 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:57:35.556235 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:57:35.556235 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:57:35.556235 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:57:35.556235 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:57:35.556235 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 24 05:57:35.556235 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:57:35.556235 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:57:35.556235 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:57:35.556235 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:57:35.557060 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:57:35.557060 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:57:35.557060 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:57:35.557060 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:57:35.557060 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:57:35.557060 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:57:35.557060 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:57:35.557060 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:57:35.557060 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:57:35.557060 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:57:35.557060 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 05:57:35.557060 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 24 05:57:35.564454 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 245/486] 10.222.0.62 () {56 vars in 1102 bytes} [Wed Aug 24 05:57:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:57:35.566058 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68041c4-ba24-4c4e-96d0-8c04c29cef76 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:35.622860 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:35.679886 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 242/487] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Aug 24 05:57:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 05:57:35.709701 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68041c4-ba24-4c4e-96d0-8c04c29cef76 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:35.762507 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebe6f09-726a-4fb1-9fa9-1f280eb9e85d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:35.763118 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebe6f09-726a-4fb1-9fa9-1f280eb9e85d None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:35.763396 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebe6f09-726a-4fb1-9fa9-1f280eb9e85d None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:35.763668 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebe6f09-726a-4fb1-9fa9-1f280eb9e85d None admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:35.764323 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebe6f09-726a-4fb1-9fa9-1f280eb9e85d None admin] RBAC: Adding query filter params (name=service) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:57:35.764655 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebe6f09-726a-4fb1-9fa9-1f280eb9e85d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:35.766556 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebe6f09-726a-4fb1-9fa9-1f280eb9e85d None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:35.779899 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 246/488] 10.222.0.62 () {56 vars in 1103 bytes} [Wed Aug 24 05:57:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:57:35.786320 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f463b3b-92b8-40a9-946e-49b9f0587323 None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:35.896043 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f463b3b-92b8-40a9-946e-49b9f0587323 None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:35.916474 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebe6f09-726a-4fb1-9fa9-1f280eb9e85d None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:35.947768 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7660f0c-bcec-4f4d-a3c1-50c5417c1165 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:35.948398 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7660f0c-bcec-4f4d-a3c1-50c5417c1165 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:35.948753 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7660f0c-bcec-4f4d-a3c1-50c5417c1165 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:35.949027 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7660f0c-bcec-4f4d-a3c1-50c5417c1165 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:35.960953 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7660f0c-bcec-4f4d-a3c1-50c5417c1165 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:35.962476 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7660f0c-bcec-4f4d-a3c1-50c5417c1165 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:35.978165 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application [None req-c7660f0c-bcec-4f4d-a3c1-50c5417c1165 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:57:35.978165 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:57:35.978165 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:57:35.978165 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:57:35.978165 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:57:35.978165 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:57:35.978165 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:57:35.978165 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:57:35.978165 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:57:35.978165 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:57:35.978165 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:57:35.978165 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:57:35.978165 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 05:57:35.978165 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 05:57:35.978165 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 05:57:35.978165 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 05:57:35.979089 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:57:35.979089 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:57:35.979089 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:57:35.979089 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:57:35.979089 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:57:35.979089 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:57:35.979089 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:57:35.979089 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application with Lock( Aug 24 05:57:35.979089 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:57:35.979089 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:57:35.979089 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:57:35.979089 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:57:35.979089 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:57:35.979089 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:57:35.979089 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:57:35.979089 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:57:35.979089 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 05:57:35.979915 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 05:57:35.979915 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:57:35.979915 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Aug 24 05:57:35.980974 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 243/489] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Aug 24 05:57:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:57:35.981824 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7660f0c-bcec-4f4d-a3c1-50c5417c1165 None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:36.019443 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:36.030230 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:36.090175 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebe6f09-726a-4fb1-9fa9-1f280eb9e85d None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:36.111020 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7660f0c-bcec-4f4d-a3c1-50c5417c1165 None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:36.134325 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107c266e-b73e-4222-aba7-145af584fd3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:36.134998 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107c266e-b73e-4222-aba7-145af584fd3a None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:36.135283 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107c266e-b73e-4222-aba7-145af584fd3a None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:36.135555 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107c266e-b73e-4222-aba7-145af584fd3a None admin] PATH_INFO: `/v3/roles/service` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:36.148205 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107c266e-b73e-4222-aba7-145af584fd3a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:57:36.154479 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-107c266e-b73e-4222-aba7-145af584fd3a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 05:57:36.154479 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:57:36.154479 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:57:36.154479 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:57:36.154479 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:57:36.154479 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:57:36.154479 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:57:36.154479 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:57:36.154479 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:57:36.154479 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:57:36.154479 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:57:36.154479 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:57:36.154479 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:57:36.154479 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:57:36.154479 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:57:36.154479 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:57:36.156284 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:57:36.156284 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:57:36.156284 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:57:36.156284 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:57:36.156284 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:57:36.156284 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:57:36.156284 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:57:36.156284 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:57:36.156284 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:57:36.156284 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:57:36.156284 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:57:36.156284 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 24 05:57:36.156284 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:57:36.156284 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:57:36.156284 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:57:36.156284 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:57:36.158792 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:57:36.158792 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:57:36.158792 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:57:36.158792 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:57:36.158792 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:57:36.158792 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:57:36.158792 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:57:36.158792 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:57:36.158792 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:57:36.158792 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:57:36.158792 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 05:57:36.158792 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 24 05:57:36.160087 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285c7088-9ff4-4f6b-82cc-bfcbd43c0708 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:36.160087 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285c7088-9ff4-4f6b-82cc-bfcbd43c0708 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:36.160087 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285c7088-9ff4-4f6b-82cc-bfcbd43c0708 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:36.160087 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285c7088-9ff4-4f6b-82cc-bfcbd43c0708 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:36.160595 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 247/490] 10.222.0.62 () {56 vars in 1102 bytes} [Wed Aug 24 05:57:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 05:57:36.160595 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107c266e-b73e-4222-aba7-145af584fd3a None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:36.168456 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285c7088-9ff4-4f6b-82cc-bfcbd43c0708 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:57:36.174497 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application [None req-285c7088-9ff4-4f6b-82cc-bfcbd43c0708 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 05:57:36.174497 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:57:36.174497 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:57:36.174497 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:57:36.174497 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:57:36.174497 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:57:36.174497 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:57:36.174497 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:57:36.174497 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:57:36.174497 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:57:36.174497 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:57:36.174497 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:57:36.174497 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:57:36.174497 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:57:36.174497 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:57:36.174497 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:57:36.176457 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:57:36.176457 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:57:36.176457 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:57:36.176457 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:57:36.176457 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:57:36.176457 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:57:36.176457 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:57:36.176457 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:57:36.176457 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:57:36.176457 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:57:36.176457 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:57:36.176457 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application with Lock( Aug 24 05:57:36.176457 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:57:36.176457 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:57:36.176457 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:57:36.176457 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:57:36.178502 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:57:36.178502 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:57:36.178502 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:57:36.178502 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:57:36.178502 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:57:36.178502 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:57:36.178502 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:57:36.178502 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:57:36.178502 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:57:36.178502 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:57:36.178502 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 05:57:36.178502 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Aug 24 05:57:36.178502 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 244/491] 10.222.0.62 () {56 vars in 1102 bytes} [Wed Aug 24 05:57:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 05:57:36.178502 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285c7088-9ff4-4f6b-82cc-bfcbd43c0708 None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:36.268940 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107c266e-b73e-4222-aba7-145af584fd3a None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:36.272947 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:36.285851 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:36.291079 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285c7088-9ff4-4f6b-82cc-bfcbd43c0708 None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:36.305312 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5548fbdb-aeea-4009-97ce-21d35aada484 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:36.305728 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5548fbdb-aeea-4009-97ce-21d35aada484 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:36.306012 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5548fbdb-aeea-4009-97ce-21d35aada484 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:36.306289 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5548fbdb-aeea-4009-97ce-21d35aada484 None admin] PATH_INFO: `/v3/domains` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:36.306989 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5548fbdb-aeea-4009-97ce-21d35aada484 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:57:36.307300 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5548fbdb-aeea-4009-97ce-21d35aada484 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:36.308867 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5548fbdb-aeea-4009-97ce-21d35aada484 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:36.327042 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 248/492] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Aug 24 05:57:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:57:36.327042 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5548fbdb-aeea-4009-97ce-21d35aada484 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:36.338457 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6289ca1-cc87-41a7-993c-7098721000a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:36.339066 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6289ca1-cc87-41a7-993c-7098721000a3 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:36.339345 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6289ca1-cc87-41a7-993c-7098721000a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:36.339618 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6289ca1-cc87-41a7-993c-7098721000a3 None admin] PATH_INFO: `/v3/roles` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:36.340275 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6289ca1-cc87-41a7-993c-7098721000a3 None admin] RBAC: Adding query filter params (name=service) {{(pid=81851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:57:36.341060 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6289ca1-cc87-41a7-993c-7098721000a3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:36.342526 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6289ca1-cc87-41a7-993c-7098721000a3 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:36.353615 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 245/493] 10.222.0.62 () {56 vars in 1103 bytes} [Wed Aug 24 05:57:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:57:36.354508 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6289ca1-cc87-41a7-993c-7098721000a3 None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:36.456169 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5548fbdb-aeea-4009-97ce-21d35aada484 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:36.473447 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6289ca1-cc87-41a7-993c-7098721000a3 None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:36.492460 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb97ef8c-a9ef-4fce-8a4b-9938fdd98847 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:36.493098 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb97ef8c-a9ef-4fce-8a4b-9938fdd98847 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:36.493379 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb97ef8c-a9ef-4fce-8a4b-9938fdd98847 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:36.493653 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb97ef8c-a9ef-4fce-8a4b-9938fdd98847 None admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:36.494749 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb97ef8c-a9ef-4fce-8a4b-9938fdd98847 None admin] RBAC: Adding query filter params (name=service) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:57:36.495048 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb97ef8c-a9ef-4fce-8a4b-9938fdd98847 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:36.496669 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb97ef8c-a9ef-4fce-8a4b-9938fdd98847 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:36.505368 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 249/494] 10.222.0.62 () {56 vars in 1103 bytes} [Wed Aug 24 05:57:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:57:36.506245 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb97ef8c-a9ef-4fce-8a4b-9938fdd98847 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:36.513167 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185c8e73-a235-45c4-89e3-f6ac43e1cd8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:36.513806 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185c8e73-a235-45c4-89e3-f6ac43e1cd8a None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:36.514090 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185c8e73-a235-45c4-89e3-f6ac43e1cd8a None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:36.514364 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185c8e73-a235-45c4-89e3-f6ac43e1cd8a None admin] PATH_INFO: `/v3/users/nova` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:36.522639 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185c8e73-a235-45c4-89e3-f6ac43e1cd8a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:36.524102 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185c8e73-a235-45c4-89e3-f6ac43e1cd8a None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:36.536614 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application [None req-185c8e73-a235-45c4-89e3-f6ac43e1cd8a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 05:57:36.536614 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:57:36.536614 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:57:36.536614 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:57:36.536614 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:57:36.536614 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:57:36.536614 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:57:36.536614 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:57:36.536614 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:57:36.536614 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:57:36.536614 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:57:36.536614 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:57:36.536614 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 05:57:36.536614 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 05:57:36.536614 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 05:57:36.536614 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 05:57:36.537511 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:57:36.537511 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:57:36.537511 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 05:57:36.537511 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 05:57:36.537511 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 05:57:36.537511 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 05:57:36.537511 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:57:36.537511 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:57:36.537511 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:57:36.537511 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:57:36.537511 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:57:36.537511 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application with Lock( Aug 24 05:57:36.537511 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:57:36.537511 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:57:36.537511 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:57:36.537511 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:57:36.537511 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:57:36.537511 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:57:36.538389 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:57:36.538389 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:57:36.538389 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 05:57:36.538389 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 05:57:36.538389 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 05:57:36.538389 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 05:57:36.538389 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 05:57:36.538389 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 05:57:36.538389 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 05:57:36.538389 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Aug 24 05:57:36.538389 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 246/495] 10.222.0.62 () {56 vars in 1093 bytes} [Wed Aug 24 05:57:36 2022] GET /identity/v3/users/nova => generated 82 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:57:36.538950 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1480d160-c8af-4618-87de-7095364c5905 None None] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:36.539756 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1480d160-c8af-4618-87de-7095364c5905 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:36.539756 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1480d160-c8af-4618-87de-7095364c5905 None None] PATH_INFO: `/` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:36.544544 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 247/496] 10.222.0.62 () {54 vars in 927 bytes} [Wed Aug 24 05:57:36 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:57:36.544544 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1480d160-c8af-4618-87de-7095364c5905 None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:36.556172 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:36.557751 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:36.624451 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb97ef8c-a9ef-4fce-8a4b-9938fdd98847 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:36.651320 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1480d160-c8af-4618-87de-7095364c5905 None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:36.659618 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7514ce3c-90aa-4189-9a72-e0683e700690 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:36.660258 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7514ce3c-90aa-4189-9a72-e0683e700690 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:36.660539 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7514ce3c-90aa-4189-9a72-e0683e700690 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:36.660818 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7514ce3c-90aa-4189-9a72-e0683e700690 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:36.671202 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7514ce3c-90aa-4189-9a72-e0683e700690 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:36.673234 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7514ce3c-90aa-4189-9a72-e0683e700690 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:36.686199 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9c611f-b169-4551-b3d8-b4b82565b592 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:36.686822 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9c611f-b169-4551-b3d8-b4b82565b592 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:36.687094 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9c611f-b169-4551-b3d8-b4b82565b592 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:36.687360 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9c611f-b169-4551-b3d8-b4b82565b592 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:36.690135 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-7514ce3c-90aa-4189-9a72-e0683e700690 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:57:36.690135 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:57:36.690135 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:57:36.690135 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:57:36.690135 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:57:36.690135 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:57:36.690135 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:57:36.690135 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:57:36.690135 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:57:36.690135 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:57:36.690135 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:57:36.690135 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:57:36.690135 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 05:57:36.690135 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 05:57:36.690135 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 05:57:36.690135 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 05:57:36.690998 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:57:36.690998 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:57:36.690998 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:57:36.690998 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:57:36.690998 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:57:36.690998 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:57:36.690998 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:57:36.690998 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 24 05:57:36.690998 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:57:36.690998 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:57:36.690998 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:57:36.690998 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:57:36.690998 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:57:36.690998 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:57:36.690998 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:57:36.690998 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:57:36.690998 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 05:57:36.691863 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 05:57:36.691863 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:57:36.691863 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 24 05:57:36.691863 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 250/497] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Aug 24 05:57:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:57:36.692074 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7514ce3c-90aa-4189-9a72-e0683e700690 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:36.699760 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9c611f-b169-4551-b3d8-b4b82565b592 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:36.701239 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9c611f-b169-4551-b3d8-b4b82565b592 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:36.718185 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application [None req-5a9c611f-b169-4551-b3d8-b4b82565b592 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:57:36.718185 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:57:36.718185 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:57:36.718185 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:57:36.718185 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:57:36.718185 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:57:36.718185 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:57:36.718185 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:57:36.718185 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:57:36.718185 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:57:36.718185 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:57:36.718185 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:57:36.718185 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 05:57:36.718185 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 05:57:36.718185 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 05:57:36.718185 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 05:57:36.719627 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:57:36.719627 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:57:36.719627 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:57:36.719627 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:57:36.719627 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:57:36.719627 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:57:36.719627 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:57:36.719627 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application with Lock( Aug 24 05:57:36.719627 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:57:36.719627 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:57:36.719627 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:57:36.719627 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:57:36.719627 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:57:36.719627 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:57:36.719627 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:57:36.719627 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:57:36.719627 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 05:57:36.720560 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 05:57:36.720560 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:57:36.720560 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Aug 24 05:57:36.720560 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 248/498] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Aug 24 05:57:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:57:36.721077 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0276e836-e89d-4219-b2c0-4395190dbdf6 None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:36.721360 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0276e836-e89d-4219-b2c0-4395190dbdf6 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:36.721629 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0276e836-e89d-4219-b2c0-4395190dbdf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:36.739171 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-0276e836-e89d-4219-b2c0-4395190dbdf6 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:57:36.740358 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-0276e836-e89d-4219-b2c0-4395190dbdf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:57:36.788591 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:36.795537 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:36.798094 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7514ce3c-90aa-4189-9a72-e0683e700690 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:36.806842 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 249/499] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Aug 24 05:57:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 05:57:36.808057 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed5c4ff-1ce7-4ef7-8549-de29ce2ed30b None None] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:36.808346 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed5c4ff-1ce7-4ef7-8549-de29ce2ed30b None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:36.808657 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed5c4ff-1ce7-4ef7-8549-de29ce2ed30b None None] PATH_INFO: `/` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:36.812713 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 250/500] 10.222.0.62 () {54 vars in 927 bytes} [Wed Aug 24 05:57:36 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:57:36.812713 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed5c4ff-1ce7-4ef7-8549-de29ce2ed30b None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:36.833907 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433f556c-e9a5-4502-b921-a1e11daecebd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:36.834509 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433f556c-e9a5-4502-b921-a1e11daecebd None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:36.834788 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433f556c-e9a5-4502-b921-a1e11daecebd None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:36.835061 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433f556c-e9a5-4502-b921-a1e11daecebd None admin] PATH_INFO: `/v3/users` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:36.835784 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433f556c-e9a5-4502-b921-a1e11daecebd None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:57:36.836089 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433f556c-e9a5-4502-b921-a1e11daecebd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:36.837557 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433f556c-e9a5-4502-b921-a1e11daecebd None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:36.880266 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 251/501] 10.222.0.62 () {56 vars in 1133 bytes} [Wed Aug 24 05:57:36 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:57:36.880266 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433f556c-e9a5-4502-b921-a1e11daecebd None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:36.929405 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed5c4ff-1ce7-4ef7-8549-de29ce2ed30b None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:36.965361 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d10f745-d129-4bb0-81c7-869390b3b1f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:36.965894 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d10f745-d129-4bb0-81c7-869390b3b1f9 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:36.966177 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d10f745-d129-4bb0-81c7-869390b3b1f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:36.966451 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d10f745-d129-4bb0-81c7-869390b3b1f9 None admin] PATH_INFO: `/v3/domains` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:36.967081 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d10f745-d129-4bb0-81c7-869390b3b1f9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:57:36.967382 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d10f745-d129-4bb0-81c7-869390b3b1f9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:36.968794 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d10f745-d129-4bb0-81c7-869390b3b1f9 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:36.986044 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 251/502] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Aug 24 05:57:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:57:36.986428 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433f556c-e9a5-4502-b921-a1e11daecebd None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:36.987500 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0907876e-0a5b-435f-bdda-207b748eb19f None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:36.987888 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0907876e-0a5b-435f-bdda-207b748eb19f None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:36.988065 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0907876e-0a5b-435f-bdda-207b748eb19f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:37.010155 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-0907876e-0a5b-435f-bdda-207b748eb19f None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:57:37.011335 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-0907876e-0a5b-435f-bdda-207b748eb19f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:57:37.020202 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdcc61a-89f6-4098-8041-28f40cf90139 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:37.020833 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdcc61a-89f6-4098-8041-28f40cf90139 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:37.021116 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdcc61a-89f6-4098-8041-28f40cf90139 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:37.021384 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdcc61a-89f6-4098-8041-28f40cf90139 None admin] PATH_INFO: `/v3/domains` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:37.022019 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bdcc61a-89f6-4098-8041-28f40cf90139 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:57:37.022322 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bdcc61a-89f6-4098-8041-28f40cf90139 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:37.023753 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bdcc61a-89f6-4098-8041-28f40cf90139 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:37.035419 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 252/503] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Aug 24 05:57:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:57:37.036141 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdcc61a-89f6-4098-8041-28f40cf90139 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:37.048793 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:37.068278 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:37.094740 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 252/504] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Aug 24 05:57:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 05:57:37.094740 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0907876e-0a5b-435f-bdda-207b748eb19f None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:37.138851 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdcc61a-89f6-4098-8041-28f40cf90139 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:37.174006 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3d7b9e-3e7b-485b-847f-56f20b89daff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:37.174934 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3d7b9e-3e7b-485b-847f-56f20b89daff None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:37.175412 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3d7b9e-3e7b-485b-847f-56f20b89daff None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:37.175874 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3d7b9e-3e7b-485b-847f-56f20b89daff None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:37.194530 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0907876e-0a5b-435f-bdda-207b748eb19f None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:37.230742 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22915803-ce7d-43b8-8d7b-b4ff8df42085 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:37.231312 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22915803-ce7d-43b8-8d7b-b4ff8df42085 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:37.231593 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22915803-ce7d-43b8-8d7b-b4ff8df42085 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:37.231866 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22915803-ce7d-43b8-8d7b-b4ff8df42085 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:37.238847 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22915803-ce7d-43b8-8d7b-b4ff8df42085 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:37.240289 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22915803-ce7d-43b8-8d7b-b4ff8df42085 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:37.251543 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application [None req-22915803-ce7d-43b8-8d7b-b4ff8df42085 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 24 05:57:37.251543 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:57:37.251543 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:57:37.251543 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:57:37.251543 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:57:37.251543 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:57:37.251543 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:57:37.251543 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:57:37.251543 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:57:37.251543 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:57:37.251543 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:57:37.251543 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:57:37.251543 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 05:57:37.251543 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 05:57:37.251543 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 05:57:37.251543 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 05:57:37.252395 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:57:37.252395 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:57:37.252395 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 05:57:37.252395 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 05:57:37.252395 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 05:57:37.252395 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 05:57:37.252395 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:57:37.252395 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:57:37.252395 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:57:37.252395 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:57:37.252395 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:57:37.252395 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application with Lock( Aug 24 05:57:37.252395 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:57:37.252395 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:57:37.252395 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:57:37.252395 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:57:37.252395 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:57:37.252395 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:57:37.253434 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:57:37.253434 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:57:37.253434 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 05:57:37.253434 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 05:57:37.253434 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 05:57:37.253434 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 05:57:37.253434 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 05:57:37.253434 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 05:57:37.253434 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 24 05:57:37.253434 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Aug 24 05:57:37.253434 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 253/505] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Aug 24 05:57:37 2022] GET /identity/v3/users/glance => generated 84 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:57:37.255037 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3d7b9e-3e7b-485b-847f-56f20b89daff None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:37.256675 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3d7b9e-3e7b-485b-847f-56f20b89daff None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:37.257511 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22915803-ce7d-43b8-8d7b-b4ff8df42085 None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:37.268471 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:37.305961 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:37.364456 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 253/506] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Aug 24 05:57:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 05:57:37.365467 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3d7b9e-3e7b-485b-847f-56f20b89daff None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:37.366907 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22915803-ce7d-43b8-8d7b-b4ff8df42085 None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:37.400638 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b7fa27-81a1-4af6-9db2-6b54504ea6aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:37.401261 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b7fa27-81a1-4af6-9db2-6b54504ea6aa None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:37.401540 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b7fa27-81a1-4af6-9db2-6b54504ea6aa None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:37.401810 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b7fa27-81a1-4af6-9db2-6b54504ea6aa None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:37.409602 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b7fa27-81a1-4af6-9db2-6b54504ea6aa None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:37.411040 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b7fa27-81a1-4af6-9db2-6b54504ea6aa None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:37.422105 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application [None req-62b7fa27-81a1-4af6-9db2-6b54504ea6aa None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 05:57:37.422105 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:57:37.422105 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:57:37.422105 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:57:37.422105 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:57:37.422105 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:57:37.422105 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:57:37.422105 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:57:37.422105 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:57:37.422105 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:57:37.422105 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:57:37.422105 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:57:37.422105 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 05:57:37.422105 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 05:57:37.422105 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 05:57:37.422105 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 05:57:37.423026 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:57:37.423026 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:57:37.423026 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 05:57:37.423026 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 05:57:37.423026 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 05:57:37.423026 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 05:57:37.423026 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:57:37.423026 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:57:37.423026 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:57:37.423026 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:57:37.423026 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:57:37.423026 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application with Lock( Aug 24 05:57:37.423026 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:57:37.423026 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:57:37.423026 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:57:37.423026 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:57:37.423026 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:57:37.423026 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:57:37.423907 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:57:37.423907 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:57:37.423907 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 05:57:37.423907 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 05:57:37.423907 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 05:57:37.423907 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 05:57:37.423907 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 05:57:37.423907 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 05:57:37.423907 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 05:57:37.423907 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Aug 24 05:57:37.423907 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 254/507] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Aug 24 05:57:37 2022] GET /identity/v3/users/cinder => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:57:37.423907 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b7fa27-81a1-4af6-9db2-6b54504ea6aa None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:37.467770 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3d7b9e-3e7b-485b-847f-56f20b89daff None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:37.476780 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:37.501308 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5850efd7-ed5e-435b-a4eb-a82603412193 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:37.501856 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5850efd7-ed5e-435b-a4eb-a82603412193 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:37.502127 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5850efd7-ed5e-435b-a4eb-a82603412193 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:37.502393 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5850efd7-ed5e-435b-a4eb-a82603412193 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:37.512102 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5850efd7-ed5e-435b-a4eb-a82603412193 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:57:37.517776 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-5850efd7-ed5e-435b-a4eb-a82603412193 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 05:57:37.517776 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:57:37.517776 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:57:37.517776 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:57:37.517776 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:57:37.517776 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:57:37.517776 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:57:37.517776 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:57:37.517776 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:57:37.517776 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:57:37.517776 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:57:37.517776 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:57:37.517776 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:57:37.517776 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:57:37.517776 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:57:37.517776 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:57:37.518636 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:57:37.518636 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:57:37.518636 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:57:37.518636 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:57:37.518636 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:57:37.518636 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:57:37.518636 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:57:37.518636 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:57:37.518636 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:57:37.518636 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:57:37.518636 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:57:37.518636 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 24 05:57:37.518636 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:57:37.518636 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:57:37.518636 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:57:37.518636 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:57:37.519430 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:57:37.519430 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:57:37.519430 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:57:37.519430 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:57:37.519430 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:57:37.519430 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:57:37.519430 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:57:37.519430 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:57:37.519430 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:57:37.519430 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:57:37.519430 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 05:57:37.519430 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 24 05:57:37.519430 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 254/508] 10.222.0.62 () {56 vars in 1102 bytes} [Wed Aug 24 05:57:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:57:37.520087 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5850efd7-ed5e-435b-a4eb-a82603412193 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:37.521410 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:37.524203 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b7fa27-81a1-4af6-9db2-6b54504ea6aa None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:37.558447 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f62678-b5a7-4034-af17-907ff5b1146e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:37.559052 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f62678-b5a7-4034-af17-907ff5b1146e None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:37.559330 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f62678-b5a7-4034-af17-907ff5b1146e None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:37.559602 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f62678-b5a7-4034-af17-907ff5b1146e None admin] PATH_INFO: `/v3/users` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:37.560325 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f62678-b5a7-4034-af17-907ff5b1146e None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:57:37.560663 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f62678-b5a7-4034-af17-907ff5b1146e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:37.562097 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f62678-b5a7-4034-af17-907ff5b1146e None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:37.598326 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 255/509] 10.222.0.62 () {56 vars in 1137 bytes} [Wed Aug 24 05:57:37 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:57:37.599139 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f62678-b5a7-4034-af17-907ff5b1146e None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:37.620793 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5850efd7-ed5e-435b-a4eb-a82603412193 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:37.653800 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e44ef5b-2b08-410c-9d09-2c4eb54998b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:37.654423 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e44ef5b-2b08-410c-9d09-2c4eb54998b3 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:37.654698 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e44ef5b-2b08-410c-9d09-2c4eb54998b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:37.654969 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e44ef5b-2b08-410c-9d09-2c4eb54998b3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:37.665066 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e44ef5b-2b08-410c-9d09-2c4eb54998b3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:37.666526 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e44ef5b-2b08-410c-9d09-2c4eb54998b3 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:37.680406 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-1e44ef5b-2b08-410c-9d09-2c4eb54998b3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:57:37.680406 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:57:37.680406 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:57:37.680406 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:57:37.680406 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:57:37.680406 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:57:37.680406 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:57:37.680406 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:57:37.680406 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:57:37.680406 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:57:37.680406 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:57:37.680406 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:57:37.680406 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 05:57:37.680406 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 05:57:37.680406 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 05:57:37.680406 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 05:57:37.681520 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:57:37.681520 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:57:37.681520 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:57:37.681520 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:57:37.681520 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:57:37.681520 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:57:37.681520 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:57:37.681520 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 24 05:57:37.681520 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:57:37.681520 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:57:37.681520 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:57:37.681520 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:57:37.681520 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:57:37.681520 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:57:37.681520 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:57:37.681520 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:57:37.681520 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 05:57:37.682560 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 05:57:37.682560 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:57:37.682560 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 24 05:57:37.682560 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 255/510] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Aug 24 05:57:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:57:37.682560 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e44ef5b-2b08-410c-9d09-2c4eb54998b3 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:37.701316 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f62678-b5a7-4034-af17-907ff5b1146e None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:37.705666 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:37.736216 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9375bab7-c4d8-4100-8a65-1b3abf57e13b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:37.736882 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9375bab7-c4d8-4100-8a65-1b3abf57e13b None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:37.737163 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9375bab7-c4d8-4100-8a65-1b3abf57e13b None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:37.737432 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9375bab7-c4d8-4100-8a65-1b3abf57e13b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:37.747154 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9375bab7-c4d8-4100-8a65-1b3abf57e13b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:37.748620 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9375bab7-c4d8-4100-8a65-1b3abf57e13b None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:37.769710 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application [None req-9375bab7-c4d8-4100-8a65-1b3abf57e13b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:57:37.769710 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:57:37.769710 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:57:37.769710 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:57:37.769710 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:57:37.769710 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:57:37.769710 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:57:37.769710 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:57:37.769710 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:57:37.769710 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:57:37.769710 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:57:37.769710 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:57:37.769710 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 05:57:37.769710 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 05:57:37.769710 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 05:57:37.769710 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 05:57:37.770825 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:57:37.770825 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:57:37.770825 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:57:37.770825 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:57:37.770825 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:57:37.770825 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:57:37.770825 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:57:37.770825 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application with Lock( Aug 24 05:57:37.770825 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:57:37.770825 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:57:37.770825 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:57:37.770825 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:57:37.770825 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:57:37.770825 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:57:37.770825 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:57:37.770825 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:57:37.770825 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 05:57:37.771876 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 05:57:37.771876 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:57:37.771876 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Aug 24 05:57:37.771876 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 256/511] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Aug 24 05:57:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:57:37.771876 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9375bab7-c4d8-4100-8a65-1b3abf57e13b None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:37.782178 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:37.783699 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e44ef5b-2b08-410c-9d09-2c4eb54998b3 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:37.817710 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2060a54-7069-4613-ad6d-cb422e4452c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:37.818306 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2060a54-7069-4613-ad6d-cb422e4452c2 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:37.818580 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2060a54-7069-4613-ad6d-cb422e4452c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:37.818894 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2060a54-7069-4613-ad6d-cb422e4452c2 None admin] PATH_INFO: `/v3/users` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:37.819598 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2060a54-7069-4613-ad6d-cb422e4452c2 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:57:37.819898 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2060a54-7069-4613-ad6d-cb422e4452c2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:37.821402 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2060a54-7069-4613-ad6d-cb422e4452c2 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:37.837940 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 256/512] 10.222.0.62 () {56 vars in 1137 bytes} [Wed Aug 24 05:57:37 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:57:37.838727 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2060a54-7069-4613-ad6d-cb422e4452c2 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:37.869369 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9375bab7-c4d8-4100-8a65-1b3abf57e13b None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:37.902556 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b970d7-7ff3-4e50-9910-4d0659c4fb34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:37.903232 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b970d7-7ff3-4e50-9910-4d0659c4fb34 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:37.903517 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b970d7-7ff3-4e50-9910-4d0659c4fb34 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:37.903789 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b970d7-7ff3-4e50-9910-4d0659c4fb34 None admin] PATH_INFO: `/v3/roles` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:37.904444 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b970d7-7ff3-4e50-9910-4d0659c4fb34 None admin] RBAC: Adding query filter params (name=service) {{(pid=81851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:57:37.904779 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b970d7-7ff3-4e50-9910-4d0659c4fb34 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:37.906140 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b970d7-7ff3-4e50-9910-4d0659c4fb34 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:37.913818 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 257/513] 10.222.0.62 () {56 vars in 1103 bytes} [Wed Aug 24 05:57:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:57:37.914595 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b970d7-7ff3-4e50-9910-4d0659c4fb34 None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:37.919112 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:37.937270 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2060a54-7069-4613-ad6d-cb422e4452c2 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:37.974281 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bc2bfe-298c-4004-87ce-081d81244dbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:37.974882 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bc2bfe-298c-4004-87ce-081d81244dbd None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:37.975158 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bc2bfe-298c-4004-87ce-081d81244dbd None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:37.975428 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bc2bfe-298c-4004-87ce-081d81244dbd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:37.986199 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:38.015691 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b970d7-7ff3-4e50-9910-4d0659c4fb34 None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:38.041748 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bc2bfe-298c-4004-87ce-081d81244dbd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:38.043153 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bc2bfe-298c-4004-87ce-081d81244dbd None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:38.049247 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5aa7b4-ec85-4334-81ae-3ce7effa9397 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:38.049843 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5aa7b4-ec85-4334-81ae-3ce7effa9397 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:38.050118 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5aa7b4-ec85-4334-81ae-3ce7effa9397 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:38.050385 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5aa7b4-ec85-4334-81ae-3ce7effa9397 None admin] PATH_INFO: `/v3/domains` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:38.051005 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5aa7b4-ec85-4334-81ae-3ce7effa9397 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:57:38.051300 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5aa7b4-ec85-4334-81ae-3ce7effa9397 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:38.052689 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5aa7b4-ec85-4334-81ae-3ce7effa9397 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:38.064525 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 258/514] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Aug 24 05:57:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:57:38.065255 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5aa7b4-ec85-4334-81ae-3ce7effa9397 None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:38.119407 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:38.140386 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 257/515] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Aug 24 05:57:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 05:57:38.141741 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bc2bfe-298c-4004-87ce-081d81244dbd None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:38.168683 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5aa7b4-ec85-4334-81ae-3ce7effa9397 None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:38.197666 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:38.202597 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e967aeb-2bdc-4135-b351-90e03c04e94c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:38.203208 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e967aeb-2bdc-4135-b351-90e03c04e94c None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:38.204059 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e967aeb-2bdc-4135-b351-90e03c04e94c None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:38.204328 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e967aeb-2bdc-4135-b351-90e03c04e94c None admin] PATH_INFO: `/v3/domains` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:38.205000 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e967aeb-2bdc-4135-b351-90e03c04e94c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:57:38.205297 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e967aeb-2bdc-4135-b351-90e03c04e94c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:38.206714 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e967aeb-2bdc-4135-b351-90e03c04e94c None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:38.218748 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 259/516] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Aug 24 05:57:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:57:38.219711 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e967aeb-2bdc-4135-b351-90e03c04e94c None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:38.241853 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bc2bfe-298c-4004-87ce-081d81244dbd None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:38.277350 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02df3959-b00b-41e2-b309-1cf919929a72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:38.277947 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02df3959-b00b-41e2-b309-1cf919929a72 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:38.278221 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02df3959-b00b-41e2-b309-1cf919929a72 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:38.278486 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02df3959-b00b-41e2-b309-1cf919929a72 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:38.317528 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e967aeb-2bdc-4135-b351-90e03c04e94c None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:38.323216 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:38.344741 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02df3959-b00b-41e2-b309-1cf919929a72 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:38.346211 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02df3959-b00b-41e2-b309-1cf919929a72 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:38.350348 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62668d4-bd83-42af-be71-a56465f7d6cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:38.350963 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62668d4-bd83-42af-be71-a56465f7d6cb None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:38.351236 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62668d4-bd83-42af-be71-a56465f7d6cb None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:38.351503 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62668d4-bd83-42af-be71-a56465f7d6cb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:38.361193 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62668d4-bd83-42af-be71-a56465f7d6cb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:38.362607 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62668d4-bd83-42af-be71-a56465f7d6cb None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:38.377353 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application [None req-d62668d4-bd83-42af-be71-a56465f7d6cb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:57:38.377353 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:57:38.377353 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:57:38.377353 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:57:38.377353 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:57:38.377353 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:57:38.377353 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:57:38.377353 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:57:38.377353 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:57:38.377353 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:57:38.377353 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:57:38.377353 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:57:38.377353 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 05:57:38.377353 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 05:57:38.377353 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 05:57:38.377353 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 05:57:38.378242 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:57:38.378242 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:57:38.378242 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:57:38.378242 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:57:38.378242 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:57:38.378242 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:57:38.378242 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:57:38.378242 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application with Lock( Aug 24 05:57:38.378242 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:57:38.378242 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:57:38.378242 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:57:38.378242 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:57:38.378242 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:57:38.378242 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:57:38.378242 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:57:38.378242 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:57:38.378242 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 05:57:38.379313 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 05:57:38.379313 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:57:38.379313 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Aug 24 05:57:38.379313 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 260/517] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Aug 24 05:57:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:57:38.379313 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62668d4-bd83-42af-be71-a56465f7d6cb None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:38.427035 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:38.448885 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 258/518] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Aug 24 05:57:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 05:57:38.449706 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02df3959-b00b-41e2-b309-1cf919929a72 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:38.484480 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62668d4-bd83-42af-be71-a56465f7d6cb None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:38.517789 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdf604c-9c51-4054-a550-9cc577ba54a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:38.518387 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdf604c-9c51-4054-a550-9cc577ba54a2 None admin] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:38.518666 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdf604c-9c51-4054-a550-9cc577ba54a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:38.518938 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdf604c-9c51-4054-a550-9cc577ba54a2 None admin] PATH_INFO: `/v3/users` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:38.519560 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afdf604c-9c51-4054-a550-9cc577ba54a2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:38.521025 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afdf604c-9c51-4054-a550-9cc577ba54a2 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:38.525062 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:38.545833 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02df3959-b00b-41e2-b309-1cf919929a72 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:38.580412 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4eb645f-573d-43ea-bd58-909dba65da14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:38.581036 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4eb645f-573d-43ea-bd58-909dba65da14 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:38.581313 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4eb645f-573d-43ea-bd58-909dba65da14 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:38.581579 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4eb645f-573d-43ea-bd58-909dba65da14 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:38.587292 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 261/519] 10.222.0.62 () {60 vars in 1132 bytes} [Wed Aug 24 05:57:38 2022] POST /identity/v3/users => generated 247 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:57:38.588110 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdf604c-9c51-4054-a550-9cc577ba54a2 None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:38.592287 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4eb645f-573d-43ea-bd58-909dba65da14 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:38.595855 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4eb645f-573d-43ea-bd58-909dba65da14 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:38.615708 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-c4eb645f-573d-43ea-bd58-909dba65da14 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:57:38.615708 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:57:38.615708 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:57:38.615708 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:57:38.615708 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:57:38.615708 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:57:38.615708 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:57:38.615708 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:57:38.615708 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:57:38.615708 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:57:38.615708 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:57:38.615708 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:57:38.615708 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 05:57:38.615708 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 05:57:38.615708 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 05:57:38.615708 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 05:57:38.619782 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:57:38.619782 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:57:38.619782 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:57:38.619782 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:57:38.619782 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:57:38.619782 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:57:38.619782 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:57:38.619782 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 24 05:57:38.619782 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:57:38.619782 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:57:38.619782 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:57:38.619782 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:57:38.619782 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:57:38.619782 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:57:38.619782 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:57:38.619782 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:57:38.619782 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 05:57:38.620686 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 05:57:38.620686 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:57:38.620686 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 24 05:57:38.620686 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 259/520] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Aug 24 05:57:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:57:38.620686 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4eb645f-573d-43ea-bd58-909dba65da14 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:38.696853 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:38.700100 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdf604c-9c51-4054-a550-9cc577ba54a2 None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:38.740902 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae5022b-482d-4d9f-85b3-35f691197a1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:38.741526 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae5022b-482d-4d9f-85b3-35f691197a1b None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:38.741804 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae5022b-482d-4d9f-85b3-35f691197a1b None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:38.742070 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae5022b-482d-4d9f-85b3-35f691197a1b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:38.748863 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae5022b-482d-4d9f-85b3-35f691197a1b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:38.750890 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae5022b-482d-4d9f-85b3-35f691197a1b None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:38.751505 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4eb645f-573d-43ea-bd58-909dba65da14 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:38.765919 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application [None req-cae5022b-482d-4d9f-85b3-35f691197a1b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 05:57:38.765919 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:57:38.765919 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:57:38.765919 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:57:38.765919 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:57:38.765919 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:57:38.765919 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:57:38.765919 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:57:38.765919 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:57:38.765919 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:57:38.765919 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:57:38.765919 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:57:38.765919 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 05:57:38.765919 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 05:57:38.765919 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 05:57:38.766957 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 05:57:38.766957 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:57:38.766957 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:57:38.766957 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:57:38.766957 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:57:38.766957 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:57:38.766957 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:57:38.766957 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:57:38.766957 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application with Lock( Aug 24 05:57:38.766957 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:57:38.766957 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:57:38.766957 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:57:38.766957 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:57:38.766957 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:57:38.766957 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:57:38.766957 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:57:38.766957 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:57:38.767939 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 05:57:38.767939 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 05:57:38.767939 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 05:57:38.767939 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 05:57:38.767939 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 05:57:38.767939 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 05:57:38.767939 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 05:57:38.767939 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Aug 24 05:57:38.767939 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 262/521] 10.222.0.62 () {56 vars in 1111 bytes} [Wed Aug 24 05:57:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:57:38.768402 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae5022b-482d-4d9f-85b3-35f691197a1b None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:38.792725 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:38.801846 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2825655-9a92-4f55-81c0-e4ba719ad808 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:38.802454 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2825655-9a92-4f55-81c0-e4ba719ad808 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:38.802884 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2825655-9a92-4f55-81c0-e4ba719ad808 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:38.803182 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2825655-9a92-4f55-81c0-e4ba719ad808 None admin] PATH_INFO: `/v3/domains` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:38.804126 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2825655-9a92-4f55-81c0-e4ba719ad808 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:57:38.804803 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2825655-9a92-4f55-81c0-e4ba719ad808 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:38.806184 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2825655-9a92-4f55-81c0-e4ba719ad808 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:38.826176 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 260/522] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Aug 24 05:57:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:57:38.827063 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2825655-9a92-4f55-81c0-e4ba719ad808 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:38.882350 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae5022b-482d-4d9f-85b3-35f691197a1b None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:38.917906 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d634b9-2903-464b-bfe3-bcbee332a7be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:38.918532 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d634b9-2903-464b-bfe3-bcbee332a7be None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:38.918927 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d634b9-2903-464b-bfe3-bcbee332a7be None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:38.919203 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d634b9-2903-464b-bfe3-bcbee332a7be None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:38.931477 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2825655-9a92-4f55-81c0-e4ba719ad808 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:38.932931 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d634b9-2903-464b-bfe3-bcbee332a7be None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:38.934509 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d634b9-2903-464b-bfe3-bcbee332a7be None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:38.949822 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application [None req-03d634b9-2903-464b-bfe3-bcbee332a7be None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:57:38.949822 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:57:38.949822 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:57:38.949822 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:57:38.949822 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:57:38.949822 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:57:38.949822 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:57:38.949822 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:57:38.949822 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:57:38.949822 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:57:38.949822 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:57:38.949822 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:57:38.949822 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 05:57:38.949822 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 05:57:38.949822 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 05:57:38.949822 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 05:57:38.950818 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:57:38.950818 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:57:38.950818 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:57:38.950818 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:57:38.950818 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:57:38.950818 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:57:38.950818 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:57:38.950818 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application with Lock( Aug 24 05:57:38.950818 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:57:38.950818 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:57:38.950818 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:57:38.950818 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:57:38.950818 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:57:38.950818 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:57:38.950818 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:57:38.950818 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:57:38.950818 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 05:57:38.951670 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 05:57:38.951670 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:57:38.951670 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Aug 24 05:57:38.956056 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 263/523] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Aug 24 05:57:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:57:38.956914 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d634b9-2903-464b-bfe3-bcbee332a7be None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:38.967648 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8350cac9-3957-4794-809f-482e88948dd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:38.968257 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8350cac9-3957-4794-809f-482e88948dd7 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:38.968570 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8350cac9-3957-4794-809f-482e88948dd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:38.968887 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8350cac9-3957-4794-809f-482e88948dd7 None admin] PATH_INFO: `/v3/domains` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:38.969532 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8350cac9-3957-4794-809f-482e88948dd7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:57:38.969837 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8350cac9-3957-4794-809f-482e88948dd7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:38.971221 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8350cac9-3957-4794-809f-482e88948dd7 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:38.972626 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:38.983350 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 261/524] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Aug 24 05:57:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:57:38.984332 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8350cac9-3957-4794-809f-482e88948dd7 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:39.040800 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:39.070934 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d634b9-2903-464b-bfe3-bcbee332a7be None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:39.099107 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8350cac9-3957-4794-809f-482e88948dd7 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:39.109684 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42a3293-698d-431f-87ca-97ec8dbb5249 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:39.110295 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42a3293-698d-431f-87ca-97ec8dbb5249 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:39.110577 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42a3293-698d-431f-87ca-97ec8dbb5249 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:39.110849 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42a3293-698d-431f-87ca-97ec8dbb5249 None admin] PATH_INFO: `/v3/projects` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:39.111536 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42a3293-698d-431f-87ca-97ec8dbb5249 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:57:39.111843 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42a3293-698d-431f-87ca-97ec8dbb5249 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:39.113307 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42a3293-698d-431f-87ca-97ec8dbb5249 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:39.132599 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 264/525] 10.222.0.62 () {56 vars in 1148 bytes} [Wed Aug 24 05:57:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:57:39.132922 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42a3293-698d-431f-87ca-97ec8dbb5249 None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:39.139526 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6658dc4c-6339-4086-8031-b747f281c863 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:39.140159 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6658dc4c-6339-4086-8031-b747f281c863 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:39.140433 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6658dc4c-6339-4086-8031-b747f281c863 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:39.140766 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6658dc4c-6339-4086-8031-b747f281c863 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:39.150246 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6658dc4c-6339-4086-8031-b747f281c863 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:39.151673 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6658dc4c-6339-4086-8031-b747f281c863 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:39.164042 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-6658dc4c-6339-4086-8031-b747f281c863 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 24 05:57:39.164042 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:57:39.164042 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:57:39.164042 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:57:39.164042 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:57:39.164042 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:57:39.164042 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:57:39.164042 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:57:39.164042 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:57:39.164042 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:57:39.164042 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:57:39.164042 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:57:39.164042 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 05:57:39.164042 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 05:57:39.164042 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 05:57:39.164042 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 05:57:39.165001 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:57:39.165001 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:57:39.165001 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 05:57:39.165001 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 05:57:39.165001 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 05:57:39.165001 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 05:57:39.165001 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:57:39.165001 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:57:39.165001 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:57:39.165001 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:57:39.165001 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:57:39.165001 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 24 05:57:39.165001 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:57:39.165001 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:57:39.165001 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:57:39.165001 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:57:39.165001 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:57:39.165001 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:57:39.166005 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:57:39.166005 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:57:39.166005 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 05:57:39.166005 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 05:57:39.166005 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 05:57:39.166005 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 05:57:39.166005 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 05:57:39.166005 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 05:57:39.166005 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 24 05:57:39.166005 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 24 05:57:39.166005 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 262/526] 10.222.0.62 () {56 vars in 1102 bytes} [Wed Aug 24 05:57:38 2022] GET /identity/v3/users/neutron => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:57:39.166005 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6658dc4c-6339-4086-8031-b747f281c863 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:39.217675 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:39.248085 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42a3293-698d-431f-87ca-97ec8dbb5249 None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:39.272769 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6658dc4c-6339-4086-8031-b747f281c863 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:39.283733 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:39.288982 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6614679-1081-421a-ae45-f7f21219dfca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:39.289977 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6614679-1081-421a-ae45-f7f21219dfca None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:39.290710 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6614679-1081-421a-ae45-f7f21219dfca None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:39.291157 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6614679-1081-421a-ae45-f7f21219dfca None admin] PATH_INFO: `/v3/domains` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:39.292019 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6614679-1081-421a-ae45-f7f21219dfca None admin] RBAC: Adding query filter params (name=Default) {{(pid=81851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:57:39.292512 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6614679-1081-421a-ae45-f7f21219dfca None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:39.297548 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6614679-1081-421a-ae45-f7f21219dfca None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:39.307239 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 265/527] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Aug 24 05:57:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:57:39.308241 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6614679-1081-421a-ae45-f7f21219dfca None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:39.314897 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d59512d-3140-48ce-bcee-41616d863186 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:39.315531 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d59512d-3140-48ce-bcee-41616d863186 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:39.315810 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d59512d-3140-48ce-bcee-41616d863186 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:39.316083 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d59512d-3140-48ce-bcee-41616d863186 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:39.324817 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d59512d-3140-48ce-bcee-41616d863186 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:39.326272 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d59512d-3140-48ce-bcee-41616d863186 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:39.338808 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-6d59512d-3140-48ce-bcee-41616d863186 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 05:57:39.338808 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:57:39.338808 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:57:39.338808 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:57:39.338808 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:57:39.338808 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:57:39.338808 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:57:39.338808 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:57:39.338808 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:57:39.338808 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:57:39.338808 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:57:39.338808 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:57:39.338808 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 05:57:39.338808 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 05:57:39.338808 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 05:57:39.339815 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 05:57:39.339815 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:57:39.339815 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:57:39.339815 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:57:39.339815 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:57:39.339815 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:57:39.339815 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:57:39.339815 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:57:39.339815 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 24 05:57:39.339815 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:57:39.339815 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:57:39.339815 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:57:39.339815 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:57:39.339815 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:57:39.339815 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:57:39.339815 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:57:39.339815 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:57:39.341159 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 05:57:39.341159 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 05:57:39.341159 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 05:57:39.341159 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 05:57:39.341159 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 05:57:39.341159 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 05:57:39.341159 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 05:57:39.341159 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 24 05:57:39.341159 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 263/528] 10.222.0.62 () {56 vars in 1111 bytes} [Wed Aug 24 05:57:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:57:39.341159 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d59512d-3140-48ce-bcee-41616d863186 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:39.421053 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6614679-1081-421a-ae45-f7f21219dfca None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:39.450630 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d59512d-3140-48ce-bcee-41616d863186 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:39.456066 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:39.460339 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f12420-a5df-4e37-9a1a-233936cabb60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:39.460983 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f12420-a5df-4e37-9a1a-233936cabb60 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:39.461619 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f12420-a5df-4e37-9a1a-233936cabb60 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:39.461898 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f12420-a5df-4e37-9a1a-233936cabb60 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:39.462932 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f12420-a5df-4e37-9a1a-233936cabb60 None admin] RBAC: Adding query filter params (role.id=5d95a5fc0f324a92905533aa8d34ac15, scope.project.id=181f4762037e454489d9552cc603179c, user.id=fb6d3743340241f98cacb9e0d063298c) {{(pid=81851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:57:39.463329 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f12420-a5df-4e37-9a1a-233936cabb60 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:39.464956 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f12420-a5df-4e37-9a1a-233936cabb60 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:39.471447 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 266/529] 10.222.0.62 () {56 vars in 1374 bytes} [Wed Aug 24 05:57:39 2022] GET /identity/v3/role_assignments?user.id=fb6d3743340241f98cacb9e0d063298c&scope.project.id=181f4762037e454489d9552cc603179c&role.id=5d95a5fc0f324a92905533aa8d34ac15 => generated 259 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:57:39.473939 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f12420-a5df-4e37-9a1a-233936cabb60 None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:39.490010 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bf1759-3df8-41f2-b714-b29d4ee81fe6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:39.490774 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bf1759-3df8-41f2-b714-b29d4ee81fe6 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:39.491126 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bf1759-3df8-41f2-b714-b29d4ee81fe6 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:39.491516 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bf1759-3df8-41f2-b714-b29d4ee81fe6 None admin] PATH_INFO: `/v3/users` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:39.492630 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88bf1759-3df8-41f2-b714-b29d4ee81fe6 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:57:39.492753 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88bf1759-3df8-41f2-b714-b29d4ee81fe6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:39.494551 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88bf1759-3df8-41f2-b714-b29d4ee81fe6 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:39.516129 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 264/530] 10.222.0.62 () {56 vars in 1139 bytes} [Wed Aug 24 05:57:39 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:57:39.517197 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bf1759-3df8-41f2-b714-b29d4ee81fe6 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:39.552437 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:39.611114 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f12420-a5df-4e37-9a1a-233936cabb60 None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:39.649818 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a2a01c-91cb-4fb7-8c29-21743267a2a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:39.650449 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a2a01c-91cb-4fb7-8c29-21743267a2a4 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:39.650723 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a2a01c-91cb-4fb7-8c29-21743267a2a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:39.650990 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a2a01c-91cb-4fb7-8c29-21743267a2a4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:39.658436 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a2a01c-91cb-4fb7-8c29-21743267a2a4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:39.659646 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bf1759-3df8-41f2-b714-b29d4ee81fe6 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:39.659872 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a2a01c-91cb-4fb7-8c29-21743267a2a4 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:39.671434 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application [None req-87a2a01c-91cb-4fb7-8c29-21743267a2a4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 05:57:39.671434 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:57:39.671434 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:57:39.671434 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:57:39.671434 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:57:39.671434 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:57:39.671434 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:57:39.671434 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:57:39.671434 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:57:39.671434 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:57:39.671434 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:57:39.671434 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:57:39.671434 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 05:57:39.671434 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 05:57:39.671434 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 05:57:39.672423 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 05:57:39.672423 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:57:39.672423 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:57:39.672423 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:57:39.672423 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:57:39.672423 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:57:39.672423 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:57:39.672423 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:57:39.672423 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application with Lock( Aug 24 05:57:39.672423 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:57:39.672423 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:57:39.672423 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:57:39.672423 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:57:39.672423 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:57:39.672423 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:57:39.672423 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:57:39.672423 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:57:39.673547 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 05:57:39.673547 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 05:57:39.673547 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 05:57:39.673547 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 05:57:39.673547 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 05:57:39.673547 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 05:57:39.673547 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 05:57:39.673547 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Aug 24 05:57:39.677200 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 267/531] 10.222.0.62 () {56 vars in 1111 bytes} [Wed Aug 24 05:57:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:57:39.678060 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a2a01c-91cb-4fb7-8c29-21743267a2a4 None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:39.702843 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca83203-424b-49a0-bdef-83e4f6365f54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:39.704839 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca83203-424b-49a0-bdef-83e4f6365f54 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:39.705310 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca83203-424b-49a0-bdef-83e4f6365f54 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:39.705793 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca83203-424b-49a0-bdef-83e4f6365f54 None admin] PATH_INFO: `/v3/projects` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:39.706691 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca83203-424b-49a0-bdef-83e4f6365f54 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:57:39.707183 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca83203-424b-49a0-bdef-83e4f6365f54 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:39.709376 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca83203-424b-49a0-bdef-83e4f6365f54 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:39.740252 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 265/532] 10.222.0.62 () {56 vars in 1148 bytes} [Wed Aug 24 05:57:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:57:39.741428 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca83203-424b-49a0-bdef-83e4f6365f54 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:39.749362 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:39.796760 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a2a01c-91cb-4fb7-8c29-21743267a2a4 None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:39.833249 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59049132-f6b9-499e-9724-ff67e4342e59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:39.833822 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59049132-f6b9-499e-9724-ff67e4342e59 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:39.834098 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59049132-f6b9-499e-9724-ff67e4342e59 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:39.834366 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59049132-f6b9-499e-9724-ff67e4342e59 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:39.863568 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:39.875853 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca83203-424b-49a0-bdef-83e4f6365f54 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:39.905487 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59049132-f6b9-499e-9724-ff67e4342e59 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:39.907601 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59049132-f6b9-499e-9724-ff67e4342e59 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:39.911381 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e5ab78-f688-4539-b015-134b28b75fd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:39.911995 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e5ab78-f688-4539-b015-134b28b75fd5 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:39.912274 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e5ab78-f688-4539-b015-134b28b75fd5 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:39.912570 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e5ab78-f688-4539-b015-134b28b75fd5 None admin] PATH_INFO: `/v3/projects` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:39.913272 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e5ab78-f688-4539-b015-134b28b75fd5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:57:39.913567 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e5ab78-f688-4539-b015-134b28b75fd5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:39.915012 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e5ab78-f688-4539-b015-134b28b75fd5 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:39.928675 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 266/533] 10.222.0.62 () {56 vars in 1148 bytes} [Wed Aug 24 05:57:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:57:39.929268 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e5ab78-f688-4539-b015-134b28b75fd5 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:39.970234 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:40.018686 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 268/534] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Aug 24 05:57:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 05:57:40.019546 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59049132-f6b9-499e-9724-ff67e4342e59 None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:40.058674 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e5ab78-f688-4539-b015-134b28b75fd5 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:40.102455 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b073bdc0-7f08-4750-9bfe-54f153db4f2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:40.103448 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b073bdc0-7f08-4750-9bfe-54f153db4f2b None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:40.103796 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b073bdc0-7f08-4750-9bfe-54f153db4f2b None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:40.104371 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b073bdc0-7f08-4750-9bfe-54f153db4f2b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:40.105671 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b073bdc0-7f08-4750-9bfe-54f153db4f2b None admin] RBAC: Adding query filter params (role.id=5d95a5fc0f324a92905533aa8d34ac15, scope.project.id=181f4762037e454489d9552cc603179c, user.id=c7d52162cbdc4218b577ecc1e69a88c9) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:57:40.106052 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b073bdc0-7f08-4750-9bfe-54f153db4f2b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:40.108019 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b073bdc0-7f08-4750-9bfe-54f153db4f2b None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:40.120614 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 267/535] 10.222.0.62 () {56 vars in 1374 bytes} [Wed Aug 24 05:57:39 2022] GET /identity/v3/role_assignments?user.id=c7d52162cbdc4218b577ecc1e69a88c9&scope.project.id=181f4762037e454489d9552cc603179c&role.id=5d95a5fc0f324a92905533aa8d34ac15 => generated 259 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:57:40.121679 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b073bdc0-7f08-4750-9bfe-54f153db4f2b None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:40.141193 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59049132-f6b9-499e-9724-ff67e4342e59 None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:40.155785 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:40.202633 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8693319-875b-4dd7-af75-8a90363d8406 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:40.203746 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8693319-875b-4dd7-af75-8a90363d8406 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:40.204019 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8693319-875b-4dd7-af75-8a90363d8406 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:40.204283 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8693319-875b-4dd7-af75-8a90363d8406 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:40.208345 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8693319-875b-4dd7-af75-8a90363d8406 None admin] RBAC: Adding query filter params (role.id=5d95a5fc0f324a92905533aa8d34ac15, scope.project.id=181f4762037e454489d9552cc603179c, user.id=8c4def55ac0e48b29836bc5c64bc7402) {{(pid=81851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:57:40.208757 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8693319-875b-4dd7-af75-8a90363d8406 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:40.210257 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8693319-875b-4dd7-af75-8a90363d8406 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:40.223575 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 269/536] 10.222.0.62 () {56 vars in 1374 bytes} [Wed Aug 24 05:57:40 2022] GET /identity/v3/role_assignments?user.id=8c4def55ac0e48b29836bc5c64bc7402&scope.project.id=181f4762037e454489d9552cc603179c&role.id=5d95a5fc0f324a92905533aa8d34ac15 => generated 259 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:57:40.282848 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b073bdc0-7f08-4750-9bfe-54f153db4f2b None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:40.318648 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e417edab-eb60-4d9c-97fe-d069cda5a524 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:40.319294 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e417edab-eb60-4d9c-97fe-d069cda5a524 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:40.319569 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e417edab-eb60-4d9c-97fe-d069cda5a524 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:40.319839 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e417edab-eb60-4d9c-97fe-d069cda5a524 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:40.337139 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e417edab-eb60-4d9c-97fe-d069cda5a524 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:40.338663 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e417edab-eb60-4d9c-97fe-d069cda5a524 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:40.361023 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-e417edab-eb60-4d9c-97fe-d069cda5a524 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:57:40.361023 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:57:40.361023 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:57:40.361023 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:57:40.361023 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:57:40.361023 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:57:40.361023 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:57:40.361023 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:57:40.361023 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:57:40.361023 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:57:40.361023 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:57:40.361023 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:57:40.361023 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 05:57:40.361023 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 05:57:40.361023 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 05:57:40.361023 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 05:57:40.363293 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:57:40.363293 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:57:40.363293 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:57:40.363293 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:57:40.363293 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:57:40.363293 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:57:40.363293 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:57:40.363293 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 24 05:57:40.363293 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:57:40.363293 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:57:40.363293 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:57:40.363293 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:57:40.363293 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:57:40.363293 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:57:40.363293 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:57:40.363293 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:57:40.363293 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 05:57:40.365423 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 05:57:40.365423 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:57:40.365423 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 24 05:57:40.365423 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 268/537] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Aug 24 05:57:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:57:40.369822 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8693319-875b-4dd7-af75-8a90363d8406 None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:40.395267 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:40.484083 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8693319-875b-4dd7-af75-8a90363d8406 None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:40.521163 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc81bc1c-1941-4e98-84f8-72af08212a0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:40.521771 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc81bc1c-1941-4e98-84f8-72af08212a0b None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:40.522041 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc81bc1c-1941-4e98-84f8-72af08212a0b None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:40.522309 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc81bc1c-1941-4e98-84f8-72af08212a0b None admin] PATH_INFO: `/v3/domains` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:40.523118 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc81bc1c-1941-4e98-84f8-72af08212a0b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:57:40.523425 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc81bc1c-1941-4e98-84f8-72af08212a0b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:40.524805 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc81bc1c-1941-4e98-84f8-72af08212a0b None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:40.537227 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 270/538] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Aug 24 05:57:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:57:40.545497 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e417edab-eb60-4d9c-97fe-d069cda5a524 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:40.588593 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5dfe68-a242-40ee-902c-a933509320bf None None] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:40.588901 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5dfe68-a242-40ee-902c-a933509320bf None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:40.589216 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5dfe68-a242-40ee-902c-a933509320bf None None] PATH_INFO: `/` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:40.590539 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 271/539] 10.222.0.62 () {54 vars in 927 bytes} [Wed Aug 24 05:57:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 05:57:40.596719 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c40a2a2-a612-4843-90a3-15f980769d2b None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:40.597005 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c40a2a2-a612-4843-90a3-15f980769d2b None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:40.597276 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c40a2a2-a612-4843-90a3-15f980769d2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:40.618246 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-9c40a2a2-a612-4843-90a3-15f980769d2b None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:57:40.620348 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-9c40a2a2-a612-4843-90a3-15f980769d2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:57:40.634128 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:40.716535 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e417edab-eb60-4d9c-97fe-d069cda5a524 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:40.743878 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 272/540] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Aug 24 05:57:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 05:57:40.767206 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9da41bc-a8a5-4e27-ac5a-ce930cc663c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:40.767846 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9da41bc-a8a5-4e27-ac5a-ce930cc663c1 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:40.768124 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9da41bc-a8a5-4e27-ac5a-ce930cc663c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:40.768396 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9da41bc-a8a5-4e27-ac5a-ce930cc663c1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:40.781775 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9da41bc-a8a5-4e27-ac5a-ce930cc663c1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:40.783347 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9da41bc-a8a5-4e27-ac5a-ce930cc663c1 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:40.802556 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-f9da41bc-a8a5-4e27-ac5a-ce930cc663c1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 05:57:40.802556 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:57:40.802556 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:57:40.802556 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:57:40.802556 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:57:40.802556 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:57:40.802556 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:57:40.802556 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:57:40.802556 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:57:40.802556 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:57:40.802556 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:57:40.802556 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:57:40.802556 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 05:57:40.802556 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 05:57:40.802556 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 05:57:40.803459 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 05:57:40.803459 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:57:40.803459 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:57:40.803459 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:57:40.803459 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:57:40.803459 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:57:40.803459 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:57:40.803459 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:57:40.803459 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 24 05:57:40.803459 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:57:40.803459 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:57:40.803459 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:57:40.803459 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:57:40.803459 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:57:40.803459 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:57:40.803459 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:57:40.803459 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:57:40.804346 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 05:57:40.804346 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 05:57:40.804346 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 05:57:40.804346 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 05:57:40.804346 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 05:57:40.804346 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 05:57:40.804346 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 05:57:40.804346 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 24 05:57:40.804346 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 269/541] 10.222.0.62 () {56 vars in 1111 bytes} [Wed Aug 24 05:57:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:57:40.809460 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c40a2a2-a612-4843-90a3-15f980769d2b None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:40.827319 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:40.947209 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c40a2a2-a612-4843-90a3-15f980769d2b None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:40.993020 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b57c04-b903-41f5-b8d2-0fa5de39040f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:40.993631 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b57c04-b903-41f5-b8d2-0fa5de39040f None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:40.993960 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b57c04-b903-41f5-b8d2-0fa5de39040f None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:40.994180 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b57c04-b903-41f5-b8d2-0fa5de39040f None admin] PATH_INFO: `/v3/projects` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:40.994860 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b57c04-b903-41f5-b8d2-0fa5de39040f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:57:40.995159 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b57c04-b903-41f5-b8d2-0fa5de39040f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:40.996600 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b57c04-b903-41f5-b8d2-0fa5de39040f None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:41.014579 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 273/542] 10.222.0.62 () {56 vars in 1148 bytes} [Wed Aug 24 05:57:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:57:41.021048 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9da41bc-a8a5-4e27-ac5a-ce930cc663c1 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:41.144424 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9da41bc-a8a5-4e27-ac5a-ce930cc663c1 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:41.168302 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:41.177467 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b57c04-b903-41f5-b8d2-0fa5de39040f None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:41.188292 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119e1bf7-b671-436b-94e1-2e90e9ea2af9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:41.188856 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119e1bf7-b671-436b-94e1-2e90e9ea2af9 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:41.189183 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119e1bf7-b671-436b-94e1-2e90e9ea2af9 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:41.189459 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119e1bf7-b671-436b-94e1-2e90e9ea2af9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:41.190178 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119e1bf7-b671-436b-94e1-2e90e9ea2af9 None admin] RBAC: Adding query filter params (role.id=5d95a5fc0f324a92905533aa8d34ac15, scope.project.id=181f4762037e454489d9552cc603179c, user.id=0bb6c79db75e4d169331c02133d1ec32) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:57:41.190483 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119e1bf7-b671-436b-94e1-2e90e9ea2af9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:41.191924 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119e1bf7-b671-436b-94e1-2e90e9ea2af9 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:41.200609 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 270/543] 10.222.0.62 () {56 vars in 1374 bytes} [Wed Aug 24 05:57:41 2022] GET /identity/v3/role_assignments?user.id=0bb6c79db75e4d169331c02133d1ec32&scope.project.id=181f4762037e454489d9552cc603179c&role.id=5d95a5fc0f324a92905533aa8d34ac15 => generated 259 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:57:41.255625 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:41.289634 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b57c04-b903-41f5-b8d2-0fa5de39040f None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:41.326721 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae2adc4-0cb9-49d0-8c9c-87b17fe966ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:41.327763 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae2adc4-0cb9-49d0-8c9c-87b17fe966ef None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:41.328853 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae2adc4-0cb9-49d0-8c9c-87b17fe966ef None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:41.329329 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae2adc4-0cb9-49d0-8c9c-87b17fe966ef None admin] PATH_INFO: `/v3/roles/service` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:41.347925 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae2adc4-0cb9-49d0-8c9c-87b17fe966ef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:57:41.353691 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application [None req-bae2adc4-0cb9-49d0-8c9c-87b17fe966ef None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 05:57:41.353691 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:57:41.353691 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:57:41.353691 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:57:41.353691 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:57:41.353691 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:57:41.353691 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:57:41.353691 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:57:41.353691 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:57:41.353691 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:57:41.353691 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:57:41.353691 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:57:41.353691 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:57:41.353691 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:57:41.353691 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:57:41.353691 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:57:41.354556 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:57:41.354556 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:57:41.354556 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:57:41.354556 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:57:41.354556 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:57:41.354556 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:57:41.354556 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:57:41.354556 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:57:41.354556 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:57:41.354556 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:57:41.354556 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:57:41.354556 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application with Lock( Aug 24 05:57:41.354556 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:57:41.354556 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:57:41.354556 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:57:41.354556 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:57:41.355351 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:57:41.355351 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:57:41.355351 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:57:41.355351 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:57:41.355351 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:57:41.355351 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:57:41.355351 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:57:41.355351 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:57:41.355351 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:57:41.355351 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:57:41.355351 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 05:57:41.355351 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Aug 24 05:57:41.357963 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 274/544] 10.222.0.62 () {56 vars in 1102 bytes} [Wed Aug 24 05:57:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:57:41.364401 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119e1bf7-b671-436b-94e1-2e90e9ea2af9 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:41.489779 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119e1bf7-b671-436b-94e1-2e90e9ea2af9 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:41.535215 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190b06fa-d441-4e07-a7c8-ddbdabb4a404 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:41.536162 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190b06fa-d441-4e07-a7c8-ddbdabb4a404 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:41.536703 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190b06fa-d441-4e07-a7c8-ddbdabb4a404 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:41.537172 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190b06fa-d441-4e07-a7c8-ddbdabb4a404 None admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:41.538025 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190b06fa-d441-4e07-a7c8-ddbdabb4a404 None admin] RBAC: Adding query filter params (name=service) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:57:41.538543 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190b06fa-d441-4e07-a7c8-ddbdabb4a404 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:41.540104 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190b06fa-d441-4e07-a7c8-ddbdabb4a404 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:41.549936 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 271/545] 10.222.0.62 () {56 vars in 1103 bytes} [Wed Aug 24 05:57:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:57:41.555740 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae2adc4-0cb9-49d0-8c9c-87b17fe966ef None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:41.663872 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae2adc4-0cb9-49d0-8c9c-87b17fe966ef None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:41.694403 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:41.700462 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3f1e03-3627-44e1-84dd-a84cec64dc39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:41.701343 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3f1e03-3627-44e1-84dd-a84cec64dc39 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:41.702332 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3f1e03-3627-44e1-84dd-a84cec64dc39 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:41.702801 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3f1e03-3627-44e1-84dd-a84cec64dc39 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:41.716676 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3f1e03-3627-44e1-84dd-a84cec64dc39 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:41.718399 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3f1e03-3627-44e1-84dd-a84cec64dc39 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:41.733023 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application [None req-7e3f1e03-3627-44e1-84dd-a84cec64dc39 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:57:41.733023 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:57:41.733023 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:57:41.733023 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:57:41.733023 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:57:41.733023 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:57:41.733023 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:57:41.733023 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:57:41.733023 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:57:41.733023 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:57:41.733023 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:57:41.733023 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:57:41.733023 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 05:57:41.733023 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 05:57:41.733023 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 05:57:41.733023 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 05:57:41.733950 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:57:41.733950 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:57:41.733950 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:57:41.733950 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:57:41.733950 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:57:41.733950 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:57:41.733950 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:57:41.733950 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application with Lock( Aug 24 05:57:41.733950 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:57:41.733950 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:57:41.733950 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:57:41.733950 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:57:41.733950 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:57:41.733950 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:57:41.733950 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:57:41.733950 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:57:41.733950 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 05:57:41.735405 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 05:57:41.735405 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:57:41.735405 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Aug 24 05:57:41.737696 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 275/546] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Aug 24 05:57:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:57:41.745581 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190b06fa-d441-4e07-a7c8-ddbdabb4a404 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:41.762399 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8553d6a-5d18-4173-b31c-c83f48329a36 None None] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:41.763030 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8553d6a-5d18-4173-b31c-c83f48329a36 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:41.763316 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8553d6a-5d18-4173-b31c-c83f48329a36 None None] PATH_INFO: `/` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:41.767926 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 276/547] 10.222.0.62 () {54 vars in 927 bytes} [Wed Aug 24 05:57:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:57:41.771732 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d09c39-f488-4d7b-8a01-f1df5e22a4c4 None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:41.772012 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d09c39-f488-4d7b-8a01-f1df5e22a4c4 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:41.772280 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d09c39-f488-4d7b-8a01-f1df5e22a4c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:41.804134 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-84d09c39-f488-4d7b-8a01-f1df5e22a4c4 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:57:41.806445 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-84d09c39-f488-4d7b-8a01-f1df5e22a4c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:57:41.823537 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:41.925763 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190b06fa-d441-4e07-a7c8-ddbdabb4a404 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:41.929956 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 277/548] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Aug 24 05:57:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 05:57:41.983528 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abea9a4-8bdf-4008-ae01-aa967c4ceae2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:41.984165 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abea9a4-8bdf-4008-ae01-aa967c4ceae2 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:41.985081 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abea9a4-8bdf-4008-ae01-aa967c4ceae2 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:41.985304 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abea9a4-8bdf-4008-ae01-aa967c4ceae2 None admin] PATH_INFO: `/v3/domains` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:41.985942 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abea9a4-8bdf-4008-ae01-aa967c4ceae2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:57:41.986241 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abea9a4-8bdf-4008-ae01-aa967c4ceae2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:41.987606 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abea9a4-8bdf-4008-ae01-aa967c4ceae2 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:42.009782 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 272/549] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Aug 24 05:57:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:57:42.014996 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d09c39-f488-4d7b-8a01-f1df5e22a4c4 None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:42.126693 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d09c39-f488-4d7b-8a01-f1df5e22a4c4 None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:42.141076 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:42.162702 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a292b4-6380-4c62-aa8e-7d26c94ca06a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:42.163332 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a292b4-6380-4c62-aa8e-7d26c94ca06a None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:42.163610 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a292b4-6380-4c62-aa8e-7d26c94ca06a None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:42.166863 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a292b4-6380-4c62-aa8e-7d26c94ca06a None admin] PATH_INFO: `/v3/users/swift` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:42.171561 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a292b4-6380-4c62-aa8e-7d26c94ca06a None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:42.173610 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a292b4-6380-4c62-aa8e-7d26c94ca06a None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:42.185644 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application [None req-51a292b4-6380-4c62-aa8e-7d26c94ca06a None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 24 05:57:42.185644 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:57:42.185644 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:57:42.185644 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:57:42.185644 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:57:42.185644 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:57:42.185644 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:57:42.185644 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:57:42.185644 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:57:42.185644 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:57:42.185644 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:57:42.185644 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:57:42.185644 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 05:57:42.185644 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 05:57:42.185644 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 05:57:42.185644 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 05:57:42.187023 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:57:42.187023 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:57:42.187023 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 05:57:42.187023 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 05:57:42.187023 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 05:57:42.187023 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 05:57:42.187023 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:57:42.187023 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:57:42.187023 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:57:42.187023 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:57:42.187023 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:57:42.187023 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application with Lock( Aug 24 05:57:42.187023 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:57:42.187023 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:57:42.187023 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:57:42.187023 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:57:42.187023 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:57:42.187023 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:57:42.188430 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:57:42.188430 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:57:42.188430 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 05:57:42.188430 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 05:57:42.188430 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 05:57:42.188430 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 05:57:42.188430 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 05:57:42.188430 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 05:57:42.188430 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 24 05:57:42.188430 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Aug 24 05:57:42.188430 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 278/550] 10.222.0.62 () {56 vars in 1096 bytes} [Wed Aug 24 05:57:42 2022] GET /identity/v3/users/swift => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:57:42.197600 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abea9a4-8bdf-4008-ae01-aa967c4ceae2 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:42.277022 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a292b4-6380-4c62-aa8e-7d26c94ca06a None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:42.359574 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abea9a4-8bdf-4008-ae01-aa967c4ceae2 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:42.382569 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:42.431104 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9fca85-cec5-4018-b129-d6a38151dbf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:42.431723 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9fca85-cec5-4018-b129-d6a38151dbf9 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:42.432002 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9fca85-cec5-4018-b129-d6a38151dbf9 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:42.432272 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9fca85-cec5-4018-b129-d6a38151dbf9 None admin] PATH_INFO: `/v3/users` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:42.433026 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9fca85-cec5-4018-b129-d6a38151dbf9 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:57:42.433333 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9fca85-cec5-4018-b129-d6a38151dbf9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:42.434760 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9fca85-cec5-4018-b129-d6a38151dbf9 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:42.435069 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a292b4-6380-4c62-aa8e-7d26c94ca06a None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:42.464145 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 273/551] 10.222.0.62 () {56 vars in 1135 bytes} [Wed Aug 24 05:57:42 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:57:42.470553 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9fca85-cec5-4018-b129-d6a38151dbf9 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:42.472279 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b298db03-b116-4f2b-b5f3-4397a42e4888 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:42.472957 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b298db03-b116-4f2b-b5f3-4397a42e4888 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:42.473236 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b298db03-b116-4f2b-b5f3-4397a42e4888 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:42.473509 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b298db03-b116-4f2b-b5f3-4397a42e4888 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:42.484384 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b298db03-b116-4f2b-b5f3-4397a42e4888 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:57:42.489571 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application [None req-b298db03-b116-4f2b-b5f3-4397a42e4888 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 05:57:42.489571 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:57:42.489571 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:57:42.489571 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:57:42.489571 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:57:42.489571 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:57:42.489571 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:57:42.489571 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:57:42.489571 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:57:42.489571 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:57:42.489571 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:57:42.489571 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:57:42.489571 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:57:42.489571 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:57:42.489571 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:57:42.489571 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:57:42.490485 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:57:42.490485 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:57:42.490485 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:57:42.490485 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:57:42.490485 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:57:42.490485 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:57:42.490485 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:57:42.490485 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:57:42.490485 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:57:42.490485 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:57:42.490485 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:57:42.490485 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application with Lock( Aug 24 05:57:42.490485 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:57:42.490485 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:57:42.490485 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:57:42.490485 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:57:42.491678 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:57:42.491678 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:57:42.491678 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:57:42.491678 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:57:42.491678 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:57:42.491678 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:57:42.491678 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:57:42.491678 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:57:42.491678 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:57:42.491678 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:57:42.491678 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 05:57:42.491678 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Aug 24 05:57:42.491678 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 279/552] 10.222.0.62 () {56 vars in 1102 bytes} [Wed Aug 24 05:57:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 05:57:42.492349 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee629d1-1233-4ac1-b3bf-b98cb710c4a5 None None] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:42.492414 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee629d1-1233-4ac1-b3bf-b98cb710c4a5 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:42.497250 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee629d1-1233-4ac1-b3bf-b98cb710c4a5 None None] PATH_INFO: `/` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:42.498770 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 280/553] 10.222.0.62 () {54 vars in 927 bytes} [Wed Aug 24 05:57:42 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:57:42.499545 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee629d1-1233-4ac1-b3bf-b98cb710c4a5 None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:42.533376 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:42.581857 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9fca85-cec5-4018-b129-d6a38151dbf9 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:42.612155 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee629d1-1233-4ac1-b3bf-b98cb710c4a5 None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:42.617840 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5903f16e-6cf4-4e86-979e-4de89779fbd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:42.618804 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5903f16e-6cf4-4e86-979e-4de89779fbd6 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:42.619279 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5903f16e-6cf4-4e86-979e-4de89779fbd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:42.619744 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5903f16e-6cf4-4e86-979e-4de89779fbd6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:42.656813 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:42.660339 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da443f70-a89c-4586-9301-1ac02685412c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:42.660996 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da443f70-a89c-4586-9301-1ac02685412c None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:42.661275 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da443f70-a89c-4586-9301-1ac02685412c None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:42.661549 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da443f70-a89c-4586-9301-1ac02685412c None admin] PATH_INFO: `/v3/roles` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:42.662211 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da443f70-a89c-4586-9301-1ac02685412c None admin] RBAC: Adding query filter params (name=service) {{(pid=81851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:57:42.662515 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da443f70-a89c-4586-9301-1ac02685412c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:42.663921 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da443f70-a89c-4586-9301-1ac02685412c None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:42.675230 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 281/554] 10.222.0.62 () {56 vars in 1103 bytes} [Wed Aug 24 05:57:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:57:42.676860 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ec5fe6-ba71-4ecc-9998-2e30b758114d None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:42.677415 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ec5fe6-ba71-4ecc-9998-2e30b758114d None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:42.677981 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ec5fe6-ba71-4ecc-9998-2e30b758114d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:42.704739 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-f7ec5fe6-ba71-4ecc-9998-2e30b758114d None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:57:42.706054 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-f7ec5fe6-ba71-4ecc-9998-2e30b758114d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:57:42.709658 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5903f16e-6cf4-4e86-979e-4de89779fbd6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:42.711115 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5903f16e-6cf4-4e86-979e-4de89779fbd6 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:42.778578 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 282/555] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Aug 24 05:57:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 05:57:42.779215 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ec5fe6-ba71-4ecc-9998-2e30b758114d None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:42.810740 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:42.823169 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 274/556] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Aug 24 05:57:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 05:57:42.823169 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54661651-1bf6-4754-bea7-48d54ce02625 None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:42.823169 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54661651-1bf6-4754-bea7-48d54ce02625 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:42.823169 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54661651-1bf6-4754-bea7-48d54ce02625 None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:42.823169 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 275/557] 10.222.0.62 () {54 vars in 927 bytes} [Wed Aug 24 05:57:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:57:42.826559 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54661651-1bf6-4754-bea7-48d54ce02625 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:42.909519 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:42.923927 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ec5fe6-ba71-4ecc-9998-2e30b758114d None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:42.947281 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54661651-1bf6-4754-bea7-48d54ce02625 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:42.971841 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355a6bfb-7fe2-4cad-9a96-452ff4c18b0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:42.973148 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355a6bfb-7fe2-4cad-9a96-452ff4c18b0b None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:42.973687 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355a6bfb-7fe2-4cad-9a96-452ff4c18b0b None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:42.974223 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355a6bfb-7fe2-4cad-9a96-452ff4c18b0b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:42.984120 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a825b46a-ee3a-4705-a466-a306e6b65f74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:42.984981 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a825b46a-ee3a-4705-a466-a306e6b65f74 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:42.985615 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a825b46a-ee3a-4705-a466-a306e6b65f74 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:42.985615 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a825b46a-ee3a-4705-a466-a306e6b65f74 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:42.988003 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355a6bfb-7fe2-4cad-9a96-452ff4c18b0b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:42.990224 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355a6bfb-7fe2-4cad-9a96-452ff4c18b0b None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:42.997589 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a825b46a-ee3a-4705-a466-a306e6b65f74 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:42.999206 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a825b46a-ee3a-4705-a466-a306e6b65f74 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:43.009064 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application [None req-355a6bfb-7fe2-4cad-9a96-452ff4c18b0b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:57:43.009064 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:57:43.009064 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:57:43.009064 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:57:43.009064 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:57:43.009064 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:57:43.009064 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:57:43.009064 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:57:43.009064 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:57:43.009064 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:57:43.009064 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:57:43.009064 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:57:43.009064 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 05:57:43.009064 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 05:57:43.009064 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 05:57:43.009064 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 05:57:43.010138 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:57:43.010138 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:57:43.010138 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:57:43.010138 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:57:43.010138 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:57:43.010138 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:57:43.010138 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:57:43.010138 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application with Lock( Aug 24 05:57:43.010138 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:57:43.010138 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:57:43.010138 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:57:43.010138 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:57:43.010138 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:57:43.010138 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:57:43.010138 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:57:43.010138 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:57:43.010138 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 05:57:43.011035 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 05:57:43.011035 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:57:43.011035 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Aug 24 05:57:43.013161 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 283/558] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Aug 24 05:57:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:57:43.014513 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7f3ac6-b435-4c3f-9b90-5617c03b0cd2 None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:43.014917 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7f3ac6-b435-4c3f-9b90-5617c03b0cd2 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:43.015311 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7f3ac6-b435-4c3f-9b90-5617c03b0cd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:43.024048 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-a825b46a-ee3a-4705-a466-a306e6b65f74 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:57:43.024048 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:57:43.024048 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:57:43.024048 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:57:43.024048 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:57:43.024048 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:57:43.024048 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:57:43.024048 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:57:43.024048 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:57:43.024048 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:57:43.024048 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:57:43.024048 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:57:43.024048 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 05:57:43.024048 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 05:57:43.024048 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 05:57:43.024048 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 05:57:43.025686 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:57:43.025686 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:57:43.025686 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:57:43.025686 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:57:43.025686 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:57:43.025686 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:57:43.025686 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:57:43.025686 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 24 05:57:43.025686 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:57:43.025686 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:57:43.025686 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:57:43.025686 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:57:43.025686 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:57:43.025686 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:57:43.025686 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:57:43.025686 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:57:43.025686 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 05:57:43.026813 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 05:57:43.026813 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:57:43.026813 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 24 05:57:43.031330 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 276/559] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Aug 24 05:57:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:57:43.037530 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a825b46a-ee3a-4705-a466-a306e6b65f74 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:43.039340 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-ce7f3ac6-b435-4c3f-9b90-5617c03b0cd2 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:57:43.040723 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-ce7f3ac6-b435-4c3f-9b90-5617c03b0cd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:57:43.115849 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 284/560] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Aug 24 05:57:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 05:57:43.117503 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7f3ac6-b435-4c3f-9b90-5617c03b0cd2 None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:43.130416 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:43.146355 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a825b46a-ee3a-4705-a466-a306e6b65f74 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:43.167975 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:43.183630 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac7993e-3fab-4c6d-9692-5d64fc63dbe5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:43.184373 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac7993e-3fab-4c6d-9692-5d64fc63dbe5 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:43.184755 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac7993e-3fab-4c6d-9692-5d64fc63dbe5 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:43.185205 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac7993e-3fab-4c6d-9692-5d64fc63dbe5 None admin] PATH_INFO: `/v3/domains` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:43.186011 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac7993e-3fab-4c6d-9692-5d64fc63dbe5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:57:43.186290 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac7993e-3fab-4c6d-9692-5d64fc63dbe5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:43.188412 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac7993e-3fab-4c6d-9692-5d64fc63dbe5 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:43.204981 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 277/561] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Aug 24 05:57:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:57:43.205907 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac7993e-3fab-4c6d-9692-5d64fc63dbe5 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:43.254781 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7f3ac6-b435-4c3f-9b90-5617c03b0cd2 None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:43.292778 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2032a7-c08f-4c27-9712-4df25a4ff64e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:43.293868 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2032a7-c08f-4c27-9712-4df25a4ff64e None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:43.293868 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2032a7-c08f-4c27-9712-4df25a4ff64e None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:43.294041 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2032a7-c08f-4c27-9712-4df25a4ff64e None admin] PATH_INFO: `/v3/domains` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:43.294743 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2032a7-c08f-4c27-9712-4df25a4ff64e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:57:43.295408 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2032a7-c08f-4c27-9712-4df25a4ff64e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:43.297493 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2032a7-c08f-4c27-9712-4df25a4ff64e None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:43.314440 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 285/562] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Aug 24 05:57:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:57:43.315392 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2032a7-c08f-4c27-9712-4df25a4ff64e None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:43.331476 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac7993e-3fab-4c6d-9692-5d64fc63dbe5 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:43.375657 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ff580b-5f57-4214-894d-b73fca1de8fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:43.376726 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ff580b-5f57-4214-894d-b73fca1de8fc None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:43.377577 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ff580b-5f57-4214-894d-b73fca1de8fc None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:43.377849 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ff580b-5f57-4214-894d-b73fca1de8fc None admin] PATH_INFO: `/v3/roles/service` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:43.383782 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:43.388549 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ff580b-5f57-4214-894d-b73fca1de8fc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:57:43.393453 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-26ff580b-5f57-4214-894d-b73fca1de8fc None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 05:57:43.393453 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:57:43.393453 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:57:43.393453 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:57:43.393453 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:57:43.393453 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:57:43.393453 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:57:43.393453 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:57:43.393453 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:57:43.393453 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:57:43.393453 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:57:43.393453 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:57:43.393453 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:57:43.393453 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:57:43.393453 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:57:43.393453 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:57:43.394376 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:57:43.394376 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:57:43.394376 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:57:43.394376 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:57:43.394376 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:57:43.394376 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:57:43.394376 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:57:43.394376 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:57:43.394376 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:57:43.394376 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:57:43.394376 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:57:43.394376 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 24 05:57:43.394376 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:57:43.394376 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:57:43.394376 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:57:43.394376 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:57:43.395199 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:57:43.395199 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:57:43.395199 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:57:43.395199 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:57:43.395199 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:57:43.395199 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:57:43.395199 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:57:43.395199 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:57:43.395199 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:57:43.395199 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:57:43.395199 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 05:57:43.395199 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 24 05:57:43.395199 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 278/563] 10.222.0.62 () {56 vars in 1102 bytes} [Wed Aug 24 05:57:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:57:43.395884 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ff580b-5f57-4214-894d-b73fca1de8fc None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:43.429366 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2032a7-c08f-4c27-9712-4df25a4ff64e None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:43.441378 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:43.464628 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf13c8d-2b10-4261-b809-aa0c40b78d93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:43.466014 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf13c8d-2b10-4261-b809-aa0c40b78d93 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:43.467268 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf13c8d-2b10-4261-b809-aa0c40b78d93 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:43.467817 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf13c8d-2b10-4261-b809-aa0c40b78d93 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:43.479074 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf13c8d-2b10-4261-b809-aa0c40b78d93 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:43.481036 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf13c8d-2b10-4261-b809-aa0c40b78d93 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:43.493734 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application [None req-6bf13c8d-2b10-4261-b809-aa0c40b78d93 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 05:57:43.493734 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:57:43.493734 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:57:43.493734 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:57:43.493734 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:57:43.493734 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:57:43.493734 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:57:43.493734 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:57:43.493734 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:57:43.493734 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:57:43.493734 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:57:43.493734 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:57:43.493734 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 05:57:43.493734 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 05:57:43.493734 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 05:57:43.493734 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 05:57:43.494680 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:57:43.494680 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:57:43.494680 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 05:57:43.494680 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 05:57:43.494680 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 05:57:43.494680 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 05:57:43.494680 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:57:43.494680 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:57:43.494680 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:57:43.494680 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:57:43.494680 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:57:43.494680 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application with Lock( Aug 24 05:57:43.494680 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:57:43.494680 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:57:43.494680 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:57:43.494680 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:57:43.494680 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:57:43.494680 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:57:43.495625 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:57:43.495625 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:57:43.495625 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 05:57:43.495625 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 05:57:43.495625 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 05:57:43.495625 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 05:57:43.495625 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 05:57:43.495625 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 05:57:43.495625 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 05:57:43.495625 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Aug 24 05:57:43.495625 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 286/564] 10.222.0.62 () {56 vars in 1093 bytes} [Wed Aug 24 05:57:43 2022] GET /identity/v3/users/nova => generated 82 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:57:43.495625 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf13c8d-2b10-4261-b809-aa0c40b78d93 None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:43.503776 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ff580b-5f57-4214-894d-b73fca1de8fc None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:43.538471 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d75a26c-7f3d-4a08-a7b3-6fe0ba978fce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:43.539106 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d75a26c-7f3d-4a08-a7b3-6fe0ba978fce None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:43.539388 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d75a26c-7f3d-4a08-a7b3-6fe0ba978fce None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:43.539658 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d75a26c-7f3d-4a08-a7b3-6fe0ba978fce None admin] PATH_INFO: `/v3/projects/service` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:43.546472 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d75a26c-7f3d-4a08-a7b3-6fe0ba978fce None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:43.548664 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d75a26c-7f3d-4a08-a7b3-6fe0ba978fce None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:43.559480 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-9d75a26c-7f3d-4a08-a7b3-6fe0ba978fce None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 05:57:43.559480 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:57:43.559480 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:57:43.559480 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:57:43.559480 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:57:43.559480 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:57:43.559480 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:57:43.559480 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:57:43.559480 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:57:43.559480 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:57:43.559480 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:57:43.559480 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:57:43.559480 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 05:57:43.559480 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 05:57:43.559480 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 05:57:43.560532 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 05:57:43.560532 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:57:43.560532 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:57:43.560532 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:57:43.560532 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:57:43.560532 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:57:43.560532 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:57:43.560532 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:57:43.560532 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 24 05:57:43.560532 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:57:43.560532 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:57:43.560532 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:57:43.560532 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:57:43.560532 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:57:43.560532 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:57:43.560532 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:57:43.560532 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:57:43.561458 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 05:57:43.561458 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 05:57:43.561458 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 05:57:43.561458 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 05:57:43.561458 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 05:57:43.561458 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 05:57:43.561458 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 05:57:43.561458 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 24 05:57:43.561458 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 279/565] 10.222.0.62 () {56 vars in 1111 bytes} [Wed Aug 24 05:57:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:57:43.562967 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d75a26c-7f3d-4a08-a7b3-6fe0ba978fce None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:43.604988 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf13c8d-2b10-4261-b809-aa0c40b78d93 None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:43.629903 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:43.638294 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f856c9d6-71c4-4b8c-9f37-70dc561ac19f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:43.638907 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f856c9d6-71c4-4b8c-9f37-70dc561ac19f None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:43.639186 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f856c9d6-71c4-4b8c-9f37-70dc561ac19f None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:43.639470 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f856c9d6-71c4-4b8c-9f37-70dc561ac19f None admin] PATH_INFO: `/v3/roles/service` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:43.650537 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f856c9d6-71c4-4b8c-9f37-70dc561ac19f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:57:43.655324 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application [None req-f856c9d6-71c4-4b8c-9f37-70dc561ac19f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 05:57:43.655324 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:57:43.655324 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:57:43.655324 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:57:43.655324 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:57:43.655324 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:57:43.655324 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:57:43.655324 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:57:43.655324 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:57:43.655324 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:57:43.655324 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:57:43.655324 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:57:43.655324 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:57:43.655324 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:57:43.655324 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:57:43.655324 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:57:43.656945 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:57:43.656945 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:57:43.656945 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:57:43.656945 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:57:43.656945 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:57:43.656945 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:57:43.656945 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:57:43.656945 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:57:43.656945 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:57:43.656945 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:57:43.656945 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:57:43.656945 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application with Lock( Aug 24 05:57:43.656945 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:57:43.656945 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:57:43.656945 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:57:43.656945 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:57:43.657772 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:57:43.657772 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:57:43.657772 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:57:43.657772 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:57:43.657772 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:57:43.657772 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:57:43.657772 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:57:43.657772 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:57:43.657772 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:57:43.657772 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:57:43.657772 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 05:57:43.657772 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Aug 24 05:57:43.657772 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 287/566] 10.222.0.62 () {56 vars in 1102 bytes} [Wed Aug 24 05:57:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:57:43.658460 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f856c9d6-71c4-4b8c-9f37-70dc561ac19f None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:43.661533 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:43.664904 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d75a26c-7f3d-4a08-a7b3-6fe0ba978fce None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:43.699359 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4da294-fe57-4fab-8859-0bcb6f08e520 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:43.699893 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4da294-fe57-4fab-8859-0bcb6f08e520 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:43.700173 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4da294-fe57-4fab-8859-0bcb6f08e520 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:43.700445 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4da294-fe57-4fab-8859-0bcb6f08e520 None admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:43.701135 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4da294-fe57-4fab-8859-0bcb6f08e520 None admin] RBAC: Adding query filter params (name=service) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:57:43.701435 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4da294-fe57-4fab-8859-0bcb6f08e520 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:43.705493 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4da294-fe57-4fab-8859-0bcb6f08e520 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:43.710778 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 280/567] 10.222.0.62 () {56 vars in 1103 bytes} [Wed Aug 24 05:57:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:57:43.711959 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2655f9-119e-42a2-8b80-8fde710d85a3 None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:43.712240 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2655f9-119e-42a2-8b80-8fde710d85a3 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:43.712533 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2655f9-119e-42a2-8b80-8fde710d85a3 None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:43.713971 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 281/568] 10.222.0.62 () {54 vars in 927 bytes} [Wed Aug 24 05:57:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:57:43.714548 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2655f9-119e-42a2-8b80-8fde710d85a3 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:43.762289 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f856c9d6-71c4-4b8c-9f37-70dc561ac19f None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:43.797937 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3454e33f-ba4a-4ff6-931a-255f2cd3bb7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:43.798541 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3454e33f-ba4a-4ff6-931a-255f2cd3bb7d None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:43.798822 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3454e33f-ba4a-4ff6-931a-255f2cd3bb7d None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:43.799098 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3454e33f-ba4a-4ff6-931a-255f2cd3bb7d None admin] PATH_INFO: `/v3/users` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:43.799818 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3454e33f-ba4a-4ff6-931a-255f2cd3bb7d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:57:43.800122 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3454e33f-ba4a-4ff6-931a-255f2cd3bb7d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:43.801601 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3454e33f-ba4a-4ff6-931a-255f2cd3bb7d None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:43.813569 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2655f9-119e-42a2-8b80-8fde710d85a3 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:43.818747 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 288/569] 10.222.0.62 () {56 vars in 1133 bytes} [Wed Aug 24 05:57:43 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:57:43.819529 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3454e33f-ba4a-4ff6-931a-255f2cd3bb7d None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:43.850041 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76672b36-7956-4c1c-89f9-d8e91228d59f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:43.850641 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76672b36-7956-4c1c-89f9-d8e91228d59f None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:43.850918 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76672b36-7956-4c1c-89f9-d8e91228d59f None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:43.851193 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76672b36-7956-4c1c-89f9-d8e91228d59f None admin] PATH_INFO: `/v3/projects` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:43.851870 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76672b36-7956-4c1c-89f9-d8e91228d59f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:57:43.852178 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76672b36-7956-4c1c-89f9-d8e91228d59f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:43.853735 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76672b36-7956-4c1c-89f9-d8e91228d59f None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:43.865311 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 282/570] 10.222.0.62 () {56 vars in 1148 bytes} [Wed Aug 24 05:57:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:57:43.866027 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:43.866148 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76672b36-7956-4c1c-89f9-d8e91228d59f None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:43.888909 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:43.928136 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3454e33f-ba4a-4ff6-931a-255f2cd3bb7d None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:43.961623 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200b000a-99e7-475f-98ab-555494b59a58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:43.962162 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200b000a-99e7-475f-98ab-555494b59a58 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:43.962442 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200b000a-99e7-475f-98ab-555494b59a58 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:43.962716 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200b000a-99e7-475f-98ab-555494b59a58 None admin] PATH_INFO: `/v3/roles` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:43.962821 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76672b36-7956-4c1c-89f9-d8e91228d59f None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:43.963427 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200b000a-99e7-475f-98ab-555494b59a58 None admin] RBAC: Adding query filter params (name=service) {{(pid=81851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:57:43.963736 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200b000a-99e7-475f-98ab-555494b59a58 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:43.965152 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200b000a-99e7-475f-98ab-555494b59a58 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:43.973508 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 289/571] 10.222.0.62 () {56 vars in 1103 bytes} [Wed Aug 24 05:57:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:57:43.974652 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda69e1a-a70a-479d-a02a-e6cbad723d4d None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:43.974935 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda69e1a-a70a-479d-a02a-e6cbad723d4d None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:43.975210 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda69e1a-a70a-479d-a02a-e6cbad723d4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:43.992020 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-cda69e1a-a70a-479d-a02a-e6cbad723d4d None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:57:43.993235 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-cda69e1a-a70a-479d-a02a-e6cbad723d4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:57:43.996484 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29cbe98-e7d7-4b77-9f79-ab415bc50486 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:43.997132 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29cbe98-e7d7-4b77-9f79-ab415bc50486 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:43.997411 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29cbe98-e7d7-4b77-9f79-ab415bc50486 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:43.997683 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29cbe98-e7d7-4b77-9f79-ab415bc50486 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:44.007766 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29cbe98-e7d7-4b77-9f79-ab415bc50486 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:44.009719 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29cbe98-e7d7-4b77-9f79-ab415bc50486 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:44.024269 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-d29cbe98-e7d7-4b77-9f79-ab415bc50486 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:57:44.024269 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:57:44.024269 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:57:44.024269 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:57:44.024269 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:57:44.024269 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:57:44.024269 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:57:44.024269 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:57:44.024269 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:57:44.024269 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:57:44.024269 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:57:44.024269 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:57:44.024269 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 05:57:44.024269 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 05:57:44.024269 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 05:57:44.024269 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 05:57:44.025329 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:57:44.025329 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:57:44.025329 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:57:44.025329 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:57:44.025329 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:57:44.025329 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:57:44.025329 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:57:44.025329 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 24 05:57:44.025329 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:57:44.025329 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:57:44.025329 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:57:44.025329 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:57:44.025329 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:57:44.025329 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:57:44.025329 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:57:44.025329 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:57:44.025329 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 05:57:44.026289 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 05:57:44.026289 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:57:44.026289 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 24 05:57:44.026289 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 283/572] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Aug 24 05:57:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:57:44.026289 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29cbe98-e7d7-4b77-9f79-ab415bc50486 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:44.063766 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 290/573] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Aug 24 05:57:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 05:57:44.064639 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda69e1a-a70a-479d-a02a-e6cbad723d4d None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:44.100961 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:44.111803 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:44.125219 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29cbe98-e7d7-4b77-9f79-ab415bc50486 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:44.165727 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c54057a-52d5-4665-9c4a-c01bf62fb403 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:44.167710 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c54057a-52d5-4665-9c4a-c01bf62fb403 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:44.168203 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c54057a-52d5-4665-9c4a-c01bf62fb403 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:44.168859 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c54057a-52d5-4665-9c4a-c01bf62fb403 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:44.183614 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c54057a-52d5-4665-9c4a-c01bf62fb403 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:44.185207 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c54057a-52d5-4665-9c4a-c01bf62fb403 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:44.191305 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda69e1a-a70a-479d-a02a-e6cbad723d4d None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:44.200269 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-6c54057a-52d5-4665-9c4a-c01bf62fb403 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:57:44.200269 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:57:44.200269 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:57:44.200269 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:57:44.200269 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:57:44.200269 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:57:44.200269 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:57:44.200269 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:57:44.200269 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:57:44.200269 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:57:44.200269 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:57:44.200269 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:57:44.200269 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 05:57:44.200269 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 05:57:44.200269 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 05:57:44.200269 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 05:57:44.201410 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:57:44.201410 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:57:44.201410 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:57:44.201410 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:57:44.201410 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:57:44.201410 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:57:44.201410 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:57:44.201410 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 24 05:57:44.201410 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:57:44.201410 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:57:44.201410 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:57:44.201410 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:57:44.201410 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:57:44.201410 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:57:44.201410 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:57:44.201410 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:57:44.201410 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 05:57:44.202323 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 05:57:44.202323 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:57:44.202323 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 24 05:57:44.206896 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 284/574] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Aug 24 05:57:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:57:44.207773 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c54057a-52d5-4665-9c4a-c01bf62fb403 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:44.234208 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdefe784-d99c-42f4-9be7-b0d51e2cecfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:44.235154 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdefe784-d99c-42f4-9be7-b0d51e2cecfd None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:44.235639 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdefe784-d99c-42f4-9be7-b0d51e2cecfd None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:44.236134 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdefe784-d99c-42f4-9be7-b0d51e2cecfd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:44.237089 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdefe784-d99c-42f4-9be7-b0d51e2cecfd None admin] RBAC: Adding query filter params (role.id=5d95a5fc0f324a92905533aa8d34ac15, scope.project.id=181f4762037e454489d9552cc603179c, user.id=14d4fed04fb144d08918bd5d4f808d23) {{(pid=81851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:57:44.237589 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdefe784-d99c-42f4-9be7-b0d51e2cecfd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:44.239495 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdefe784-d99c-42f4-9be7-b0d51e2cecfd None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:44.248384 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 291/575] 10.222.0.62 () {56 vars in 1374 bytes} [Wed Aug 24 05:57:44 2022] GET /identity/v3/role_assignments?user.id=14d4fed04fb144d08918bd5d4f808d23&scope.project.id=181f4762037e454489d9552cc603179c&role.id=5d95a5fc0f324a92905533aa8d34ac15 => generated 259 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:57:44.249488 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdefe784-d99c-42f4-9be7-b0d51e2cecfd None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:44.384735 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c54057a-52d5-4665-9c4a-c01bf62fb403 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:44.386067 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:44.391334 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdefe784-d99c-42f4-9be7-b0d51e2cecfd None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:44.416281 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:44.433763 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14844f7-8e23-4530-958f-a0d72314d38e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:44.434371 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14844f7-8e23-4530-958f-a0d72314d38e None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:44.434690 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14844f7-8e23-4530-958f-a0d72314d38e None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:44.434992 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14844f7-8e23-4530-958f-a0d72314d38e None admin] PATH_INFO: `/v3/domains` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:44.435584 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14844f7-8e23-4530-958f-a0d72314d38e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:57:44.435880 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14844f7-8e23-4530-958f-a0d72314d38e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:44.436092 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9be8375-1df9-4706-b13d-34bae5a5c83e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:44.436760 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9be8375-1df9-4706-b13d-34bae5a5c83e None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:44.437040 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9be8375-1df9-4706-b13d-34bae5a5c83e None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:44.437864 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9be8375-1df9-4706-b13d-34bae5a5c83e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:44.437964 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14844f7-8e23-4530-958f-a0d72314d38e None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:44.455100 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9be8375-1df9-4706-b13d-34bae5a5c83e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:44.459876 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 292/576] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Aug 24 05:57:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:57:44.460015 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9be8375-1df9-4706-b13d-34bae5a5c83e None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:44.460120 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14844f7-8e23-4530-958f-a0d72314d38e None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:44.482322 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-e9be8375-1df9-4706-b13d-34bae5a5c83e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:57:44.482322 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:57:44.482322 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:57:44.482322 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:57:44.482322 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:57:44.482322 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:57:44.482322 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:57:44.482322 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:57:44.482322 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:57:44.482322 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:57:44.482322 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:57:44.482322 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:57:44.482322 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 05:57:44.482322 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 05:57:44.482322 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 05:57:44.482322 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 05:57:44.483311 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:57:44.483311 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:57:44.483311 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:57:44.483311 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:57:44.483311 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:57:44.483311 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:57:44.483311 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:57:44.483311 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 24 05:57:44.483311 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:57:44.483311 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:57:44.483311 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:57:44.483311 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:57:44.483311 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:57:44.483311 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:57:44.483311 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:57:44.483311 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:57:44.483311 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 05:57:44.484272 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 05:57:44.484272 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:57:44.484272 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 24 05:57:44.486349 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 285/577] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Aug 24 05:57:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:57:44.486838 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9be8375-1df9-4706-b13d-34bae5a5c83e None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:44.599475 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9be8375-1df9-4706-b13d-34bae5a5c83e None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:44.601858 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14844f7-8e23-4530-958f-a0d72314d38e None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:44.635891 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd494a35-06a5-4667-a2ed-fcde4abd9ec1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:44.636529 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd494a35-06a5-4667-a2ed-fcde4abd9ec1 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:44.636805 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd494a35-06a5-4667-a2ed-fcde4abd9ec1 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:44.636968 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9e8022-281e-4e5d-8629-655932a4ab44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:44.637102 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd494a35-06a5-4667-a2ed-fcde4abd9ec1 None admin] PATH_INFO: `/v3/domains` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:44.637600 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9e8022-281e-4e5d-8629-655932a4ab44 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:44.637699 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd494a35-06a5-4667-a2ed-fcde4abd9ec1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:57:44.637880 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9e8022-281e-4e5d-8629-655932a4ab44 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:44.637985 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd494a35-06a5-4667-a2ed-fcde4abd9ec1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:44.638149 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9e8022-281e-4e5d-8629-655932a4ab44 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:44.639329 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd494a35-06a5-4667-a2ed-fcde4abd9ec1 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:44.649178 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9e8022-281e-4e5d-8629-655932a4ab44 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:57:44.657394 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 293/578] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Aug 24 05:57:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:57:44.657394 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd494a35-06a5-4667-a2ed-fcde4abd9ec1 None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:44.657634 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-1c9e8022-281e-4e5d-8629-655932a4ab44 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 05:57:44.657634 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:57:44.657634 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:57:44.657634 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:57:44.657634 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:57:44.657634 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:57:44.657634 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:57:44.657634 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:57:44.657634 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:57:44.657634 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:57:44.657634 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:57:44.657634 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:57:44.657634 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:57:44.657634 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:57:44.657634 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:57:44.657634 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:57:44.658474 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:57:44.658474 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:57:44.658474 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:57:44.658474 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:57:44.658474 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:57:44.658474 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:57:44.658474 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:57:44.658474 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:57:44.658474 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:57:44.658474 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:57:44.658474 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:57:44.658474 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 24 05:57:44.658474 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:57:44.658474 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:57:44.658474 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:57:44.658474 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:57:44.659301 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:57:44.659301 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:57:44.659301 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:57:44.659301 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:57:44.659301 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:57:44.659301 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:57:44.659301 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:57:44.659301 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:57:44.659301 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:57:44.659301 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:57:44.659301 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 05:57:44.659301 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 24 05:57:44.659301 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 286/579] 10.222.0.62 () {56 vars in 1102 bytes} [Wed Aug 24 05:57:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:57:44.659301 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9e8022-281e-4e5d-8629-655932a4ab44 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:44.681670 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:44.706421 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:44.766328 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9e8022-281e-4e5d-8629-655932a4ab44 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:44.782961 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd494a35-06a5-4667-a2ed-fcde4abd9ec1 None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:44.801268 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26019688-d0e4-4c8e-bc37-58aa6970c300 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:44.801934 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26019688-d0e4-4c8e-bc37-58aa6970c300 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:44.802211 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26019688-d0e4-4c8e-bc37-58aa6970c300 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:44.802481 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26019688-d0e4-4c8e-bc37-58aa6970c300 None admin] PATH_INFO: `/v3/domains` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:44.803139 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26019688-d0e4-4c8e-bc37-58aa6970c300 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:57:44.803440 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26019688-d0e4-4c8e-bc37-58aa6970c300 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:44.804858 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26019688-d0e4-4c8e-bc37-58aa6970c300 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:44.815765 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 287/580] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Aug 24 05:57:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:57:44.816545 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26019688-d0e4-4c8e-bc37-58aa6970c300 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:44.872763 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04e9f54-d014-4ea3-87b3-b0b88e33c658 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:44.873391 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04e9f54-d014-4ea3-87b3-b0b88e33c658 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:44.873728 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04e9f54-d014-4ea3-87b3-b0b88e33c658 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:44.874007 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04e9f54-d014-4ea3-87b3-b0b88e33c658 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:44.881053 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04e9f54-d014-4ea3-87b3-b0b88e33c658 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:44.882493 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04e9f54-d014-4ea3-87b3-b0b88e33c658 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:44.894412 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application [None req-b04e9f54-d014-4ea3-87b3-b0b88e33c658 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 24 05:57:44.894412 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:57:44.894412 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:57:44.894412 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:57:44.894412 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:57:44.894412 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:57:44.894412 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:57:44.894412 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:57:44.894412 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:57:44.894412 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:57:44.894412 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:57:44.894412 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:57:44.894412 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 05:57:44.894412 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 05:57:44.894412 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 05:57:44.894412 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 05:57:44.895372 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:57:44.895372 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:57:44.895372 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 05:57:44.895372 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 05:57:44.895372 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 05:57:44.895372 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 05:57:44.895372 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:57:44.895372 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:57:44.895372 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:57:44.895372 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:57:44.895372 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:57:44.895372 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application with Lock( Aug 24 05:57:44.895372 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:57:44.895372 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:57:44.895372 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:57:44.895372 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:57:44.895372 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:57:44.895372 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:57:44.896383 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:57:44.896383 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:57:44.896383 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 05:57:44.896383 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 05:57:44.896383 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 05:57:44.896383 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 05:57:44.896383 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 05:57:44.896383 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 05:57:44.896383 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 24 05:57:44.896383 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Aug 24 05:57:44.896383 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 294/581] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Aug 24 05:57:44 2022] GET /identity/v3/users/glance => generated 84 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:57:44.896383 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04e9f54-d014-4ea3-87b3-b0b88e33c658 None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:44.921116 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:44.925397 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26019688-d0e4-4c8e-bc37-58aa6970c300 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:44.964117 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1621e39d-977c-4e0e-89a6-ba94da29f156 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:44.965013 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1621e39d-977c-4e0e-89a6-ba94da29f156 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:44.965462 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1621e39d-977c-4e0e-89a6-ba94da29f156 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:44.965876 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1621e39d-977c-4e0e-89a6-ba94da29f156 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:44.974768 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1621e39d-977c-4e0e-89a6-ba94da29f156 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:44.976943 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1621e39d-977c-4e0e-89a6-ba94da29f156 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:44.982072 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:44.992890 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-1621e39d-977c-4e0e-89a6-ba94da29f156 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 05:57:44.992890 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:57:44.992890 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:57:44.992890 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:57:44.992890 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:57:44.992890 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:57:44.992890 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:57:44.992890 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:57:44.992890 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:57:44.992890 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:57:44.992890 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:57:44.992890 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:57:44.992890 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 05:57:44.992890 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 05:57:44.992890 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 05:57:44.994297 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 05:57:44.994297 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:57:44.994297 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:57:44.994297 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:57:44.994297 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:57:44.994297 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:57:44.994297 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:57:44.994297 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:57:44.994297 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 24 05:57:44.994297 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:57:44.994297 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:57:44.994297 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:57:44.994297 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:57:44.994297 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:57:44.994297 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:57:44.994297 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:57:44.994297 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:57:44.995405 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 05:57:44.995405 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 05:57:44.995405 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 05:57:44.995405 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 05:57:44.995405 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 05:57:44.995405 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 05:57:44.995405 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 05:57:44.995405 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 24 05:57:45.000370 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 288/582] 10.222.0.62 () {56 vars in 1111 bytes} [Wed Aug 24 05:57:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:57:45.001215 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1621e39d-977c-4e0e-89a6-ba94da29f156 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:45.006371 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04e9f54-d014-4ea3-87b3-b0b88e33c658 None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:45.040331 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb20879-14da-4429-8975-e84ea453485c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:45.040893 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb20879-14da-4429-8975-e84ea453485c None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:45.041174 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb20879-14da-4429-8975-e84ea453485c None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:45.041445 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb20879-14da-4429-8975-e84ea453485c None admin] PATH_INFO: `/v3/roles` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:45.042097 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb20879-14da-4429-8975-e84ea453485c None admin] RBAC: Adding query filter params (name=service) {{(pid=81851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:57:45.042397 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb20879-14da-4429-8975-e84ea453485c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:45.043758 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb20879-14da-4429-8975-e84ea453485c None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:45.056150 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 295/583] 10.222.0.62 () {56 vars in 1103 bytes} [Wed Aug 24 05:57:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:57:45.056150 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb20879-14da-4429-8975-e84ea453485c None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:45.114108 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1621e39d-977c-4e0e-89a6-ba94da29f156 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:45.149705 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2855fe95-41e5-4a78-afe4-96d3ed23dda0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:45.150105 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2855fe95-41e5-4a78-afe4-96d3ed23dda0 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:45.150385 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2855fe95-41e5-4a78-afe4-96d3ed23dda0 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:45.150733 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2855fe95-41e5-4a78-afe4-96d3ed23dda0 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:45.157654 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2855fe95-41e5-4a78-afe4-96d3ed23dda0 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:45.158583 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb20879-14da-4429-8975-e84ea453485c None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:45.159095 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2855fe95-41e5-4a78-afe4-96d3ed23dda0 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:45.170598 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-2855fe95-41e5-4a78-afe4-96d3ed23dda0 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 05:57:45.170598 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:57:45.170598 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:57:45.170598 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:57:45.170598 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:57:45.170598 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:57:45.170598 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:57:45.170598 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:57:45.170598 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:57:45.170598 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:57:45.170598 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:57:45.170598 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:57:45.170598 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 05:57:45.170598 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 05:57:45.170598 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 05:57:45.170598 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 05:57:45.171612 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:57:45.171612 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:57:45.171612 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 05:57:45.171612 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 05:57:45.171612 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 05:57:45.171612 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 05:57:45.171612 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:57:45.171612 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:57:45.171612 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:57:45.171612 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:57:45.171612 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:57:45.171612 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 24 05:57:45.171612 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:57:45.171612 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:57:45.171612 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:57:45.171612 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:57:45.171612 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:57:45.171612 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:57:45.172747 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:57:45.172747 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:57:45.172747 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 05:57:45.172747 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 05:57:45.172747 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 05:57:45.172747 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 05:57:45.172747 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 05:57:45.172747 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 05:57:45.172747 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 05:57:45.172747 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 24 05:57:45.172747 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 289/584] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Aug 24 05:57:45 2022] GET /identity/v3/users/cinder => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:57:45.173356 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2855fe95-41e5-4a78-afe4-96d3ed23dda0 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:45.195674 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a94cf0-8afb-43f5-8b39-e3dbbd6ab0d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:45.196286 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a94cf0-8afb-43f5-8b39-e3dbbd6ab0d0 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:45.196571 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a94cf0-8afb-43f5-8b39-e3dbbd6ab0d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:45.196839 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a94cf0-8afb-43f5-8b39-e3dbbd6ab0d0 None admin] PATH_INFO: `/v3/users` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:45.196966 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:45.197558 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a94cf0-8afb-43f5-8b39-e3dbbd6ab0d0 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:57:45.197946 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a94cf0-8afb-43f5-8b39-e3dbbd6ab0d0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:45.199361 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a94cf0-8afb-43f5-8b39-e3dbbd6ab0d0 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:45.215536 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:45.219556 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 296/585] 10.222.0.62 () {56 vars in 1137 bytes} [Wed Aug 24 05:57:45 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:57:45.220264 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a94cf0-8afb-43f5-8b39-e3dbbd6ab0d0 None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:45.284412 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2855fe95-41e5-4a78-afe4-96d3ed23dda0 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:45.321169 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da959a87-1b9c-4438-af18-fe3637c8245c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:45.321708 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da959a87-1b9c-4438-af18-fe3637c8245c None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:45.321996 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da959a87-1b9c-4438-af18-fe3637c8245c None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:45.322267 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da959a87-1b9c-4438-af18-fe3637c8245c None admin] PATH_INFO: `/v3/projects` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:45.323067 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da959a87-1b9c-4438-af18-fe3637c8245c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:57:45.323381 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da959a87-1b9c-4438-af18-fe3637c8245c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:45.324896 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da959a87-1b9c-4438-af18-fe3637c8245c None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:45.337880 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 290/586] 10.222.0.62 () {56 vars in 1148 bytes} [Wed Aug 24 05:57:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:57:45.338737 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da959a87-1b9c-4438-af18-fe3637c8245c None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:45.344889 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a94cf0-8afb-43f5-8b39-e3dbbd6ab0d0 None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:45.380595 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88fa2cb-8711-4c18-be7d-39bcd84a6fe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:45.381228 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88fa2cb-8711-4c18-be7d-39bcd84a6fe4 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:45.381494 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88fa2cb-8711-4c18-be7d-39bcd84a6fe4 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:45.381766 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88fa2cb-8711-4c18-be7d-39bcd84a6fe4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:45.392245 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88fa2cb-8711-4c18-be7d-39bcd84a6fe4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:45.393735 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88fa2cb-8711-4c18-be7d-39bcd84a6fe4 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:45.408826 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application [None req-d88fa2cb-8711-4c18-be7d-39bcd84a6fe4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:57:45.408826 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:57:45.408826 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:57:45.408826 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:57:45.408826 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:57:45.408826 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:57:45.408826 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:57:45.408826 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:57:45.408826 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:57:45.408826 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:57:45.408826 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:57:45.408826 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:57:45.408826 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 05:57:45.408826 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 05:57:45.408826 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 05:57:45.408826 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 05:57:45.409753 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:57:45.409753 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:57:45.409753 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:57:45.409753 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:57:45.409753 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:57:45.409753 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:57:45.409753 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:57:45.409753 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application with Lock( Aug 24 05:57:45.409753 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:57:45.409753 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:57:45.409753 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:57:45.409753 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:57:45.409753 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:57:45.409753 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:57:45.409753 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:57:45.409753 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:57:45.409753 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 05:57:45.410642 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 05:57:45.410642 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:57:45.410642 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Aug 24 05:57:45.410642 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 297/587] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Aug 24 05:57:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:57:45.410642 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88fa2cb-8711-4c18-be7d-39bcd84a6fe4 None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:45.426821 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:45.448044 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da959a87-1b9c-4438-af18-fe3637c8245c None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:45.467146 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:45.481398 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b036cb-2cb8-4bf1-9b1a-51b39188f44f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:45.482000 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b036cb-2cb8-4bf1-9b1a-51b39188f44f None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:45.482276 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b036cb-2cb8-4bf1-9b1a-51b39188f44f None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:45.482541 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b036cb-2cb8-4bf1-9b1a-51b39188f44f None admin] PATH_INFO: `/v3/users` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:45.483251 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b036cb-2cb8-4bf1-9b1a-51b39188f44f None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:57:45.483551 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b036cb-2cb8-4bf1-9b1a-51b39188f44f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:45.485529 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b036cb-2cb8-4bf1-9b1a-51b39188f44f None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:45.503159 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 291/588] 10.222.0.62 () {56 vars in 1137 bytes} [Wed Aug 24 05:57:45 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:57:45.503954 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b036cb-2cb8-4bf1-9b1a-51b39188f44f None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:45.528075 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88fa2cb-8711-4c18-be7d-39bcd84a6fe4 None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:45.565215 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242e081e-1304-4cef-80cc-1e41eca5c184 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:45.565818 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242e081e-1304-4cef-80cc-1e41eca5c184 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:45.566092 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242e081e-1304-4cef-80cc-1e41eca5c184 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:45.566358 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242e081e-1304-4cef-80cc-1e41eca5c184 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:45.577234 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242e081e-1304-4cef-80cc-1e41eca5c184 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:45.578661 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242e081e-1304-4cef-80cc-1e41eca5c184 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:45.593500 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application [None req-242e081e-1304-4cef-80cc-1e41eca5c184 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:57:45.593500 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:57:45.593500 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:57:45.593500 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:57:45.593500 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:57:45.593500 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:57:45.593500 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:57:45.593500 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:57:45.593500 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:57:45.593500 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:57:45.593500 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:57:45.593500 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:57:45.593500 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 05:57:45.593500 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 05:57:45.593500 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 05:57:45.593500 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 05:57:45.594561 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:57:45.594561 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:57:45.594561 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:57:45.594561 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:57:45.594561 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:57:45.594561 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:57:45.594561 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:57:45.594561 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application with Lock( Aug 24 05:57:45.594561 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:57:45.594561 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:57:45.594561 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:57:45.594561 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:57:45.594561 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:57:45.594561 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:57:45.594561 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:57:45.594561 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:57:45.594561 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 05:57:45.595864 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 05:57:45.595864 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:57:45.595864 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Aug 24 05:57:45.595864 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 298/589] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Aug 24 05:57:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:57:45.595864 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242e081e-1304-4cef-80cc-1e41eca5c184 None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:45.611546 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b036cb-2cb8-4bf1-9b1a-51b39188f44f None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:45.647519 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c764dbf9-7887-4cc5-985e-77adebb2c7bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:45.648175 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c764dbf9-7887-4cc5-985e-77adebb2c7bb None admin] REQUEST_METHOD: `PUT` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:45.648453 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c764dbf9-7887-4cc5-985e-77adebb2c7bb None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:45.648760 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c764dbf9-7887-4cc5-985e-77adebb2c7bb None admin] PATH_INFO: `/v3/projects/181f4762037e454489d9552cc603179c/users/fb6d3743340241f98cacb9e0d063298c/roles/5d95a5fc0f324a92905533aa8d34ac15` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:45.651958 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:45.663836 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c764dbf9-7887-4cc5-985e-77adebb2c7bb None admin] RBAC: Authorizing `identity:create_grant(project_id=181f4762037e454489d9552cc603179c, user_id=fb6d3743340241f98cacb9e0d063298c, role_id=5d95a5fc0f324a92905533aa8d34ac15)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:45.665319 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c764dbf9-7887-4cc5-985e-77adebb2c7bb None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:45.702642 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242e081e-1304-4cef-80cc-1e41eca5c184 None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:45.705863 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:45.737570 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c0ce82-b09c-4044-96f1-7cef10b83a82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:45.738173 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c0ce82-b09c-4044-96f1-7cef10b83a82 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:45.738447 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c0ce82-b09c-4044-96f1-7cef10b83a82 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:45.738717 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c0ce82-b09c-4044-96f1-7cef10b83a82 None admin] PATH_INFO: `/v3/domains` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:45.739391 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c0ce82-b09c-4044-96f1-7cef10b83a82 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:57:45.739738 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c0ce82-b09c-4044-96f1-7cef10b83a82 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:45.741218 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c0ce82-b09c-4044-96f1-7cef10b83a82 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:45.743463 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 292/590] 10.222.0.62 () {58 vars in 1439 bytes} [Wed Aug 24 05:57:45 2022] PUT /identity/v3/projects/181f4762037e454489d9552cc603179c/users/fb6d3743340241f98cacb9e0d063298c/roles/5d95a5fc0f324a92905533aa8d34ac15 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:57:45.744368 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c764dbf9-7887-4cc5-985e-77adebb2c7bb None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:45.772631 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 299/591] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Aug 24 05:57:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:57:45.773889 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c0ce82-b09c-4044-96f1-7cef10b83a82 None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:45.975001 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c764dbf9-7887-4cc5-985e-77adebb2c7bb None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:45.994463 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c0ce82-b09c-4044-96f1-7cef10b83a82 None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:46.018537 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197d1baf-20ce-45c7-b0c7-6aa7308875e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:46.019543 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197d1baf-20ce-45c7-b0c7-6aa7308875e3 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:46.020515 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197d1baf-20ce-45c7-b0c7-6aa7308875e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:46.020986 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197d1baf-20ce-45c7-b0c7-6aa7308875e3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:46.036314 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197d1baf-20ce-45c7-b0c7-6aa7308875e3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:46.036989 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f430a7-ac4c-4231-9f9e-1856964c4df2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:46.037677 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f430a7-ac4c-4231-9f9e-1856964c4df2 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:46.038031 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f430a7-ac4c-4231-9f9e-1856964c4df2 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:46.038372 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f430a7-ac4c-4231-9f9e-1856964c4df2 None admin] PATH_INFO: `/v3/domains` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:46.038894 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197d1baf-20ce-45c7-b0c7-6aa7308875e3 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:46.039135 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f430a7-ac4c-4231-9f9e-1856964c4df2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:57:46.039506 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f430a7-ac4c-4231-9f9e-1856964c4df2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:46.040969 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f430a7-ac4c-4231-9f9e-1856964c4df2 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:46.054493 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 300/592] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Aug 24 05:57:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:57:46.055384 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f430a7-ac4c-4231-9f9e-1856964c4df2 None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:46.058945 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-197d1baf-20ce-45c7-b0c7-6aa7308875e3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:57:46.058945 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:57:46.058945 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:57:46.058945 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:57:46.058945 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:57:46.058945 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:57:46.058945 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:57:46.058945 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:57:46.058945 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:57:46.058945 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:57:46.058945 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:57:46.058945 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:57:46.058945 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 05:57:46.058945 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 05:57:46.058945 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 05:57:46.058945 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 05:57:46.059877 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:57:46.059877 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:57:46.059877 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:57:46.059877 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:57:46.059877 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:57:46.059877 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:57:46.059877 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:57:46.059877 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 24 05:57:46.059877 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:57:46.059877 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:57:46.059877 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:57:46.059877 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:57:46.059877 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:57:46.059877 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:57:46.059877 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:57:46.059877 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:57:46.059877 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 05:57:46.060797 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 05:57:46.060797 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:57:46.060797 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 24 05:57:46.063019 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 293/593] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Aug 24 05:57:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 318 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:57:46.063526 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197d1baf-20ce-45c7-b0c7-6aa7308875e3 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:46.092424 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:46.107253 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:46.169615 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197d1baf-20ce-45c7-b0c7-6aa7308875e3 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:46.187456 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f430a7-ac4c-4231-9f9e-1856964c4df2 None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:46.206203 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0643526-4760-4259-8477-fe4a28c0ee08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:46.206921 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0643526-4760-4259-8477-fe4a28c0ee08 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:46.207197 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0643526-4760-4259-8477-fe4a28c0ee08 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:46.207464 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0643526-4760-4259-8477-fe4a28c0ee08 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:46.214344 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0643526-4760-4259-8477-fe4a28c0ee08 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:46.216814 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0643526-4760-4259-8477-fe4a28c0ee08 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:46.223986 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccfa37e-4290-4b99-9849-91308c159545 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:46.224658 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccfa37e-4290-4b99-9849-91308c159545 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:46.224939 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccfa37e-4290-4b99-9849-91308c159545 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:46.225258 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccfa37e-4290-4b99-9849-91308c159545 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:46.232781 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ccfa37e-4290-4b99-9849-91308c159545 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:46.232983 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-b0643526-4760-4259-8477-fe4a28c0ee08 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 05:57:46.232983 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:57:46.232983 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:57:46.232983 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:57:46.232983 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:57:46.232983 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:57:46.232983 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:57:46.232983 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:57:46.232983 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:57:46.232983 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:57:46.232983 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:57:46.232983 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:57:46.232983 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 05:57:46.232983 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 05:57:46.232983 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 05:57:46.233857 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 05:57:46.233857 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:57:46.233857 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:57:46.233857 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:57:46.233857 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:57:46.233857 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:57:46.233857 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:57:46.233857 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:57:46.233857 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 24 05:57:46.233857 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:57:46.233857 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:57:46.233857 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:57:46.233857 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:57:46.233857 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:57:46.233857 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:57:46.233857 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:57:46.233857 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:57:46.234786 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 05:57:46.234786 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 05:57:46.234786 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 05:57:46.234786 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 05:57:46.234786 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 05:57:46.234786 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 05:57:46.234786 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 05:57:46.234786 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 24 05:57:46.234786 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 294/594] 10.222.0.62 () {56 vars in 1111 bytes} [Wed Aug 24 05:57:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:57:46.235265 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ccfa37e-4290-4b99-9849-91308c159545 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:46.235336 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0643526-4760-4259-8477-fe4a28c0ee08 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:46.251732 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application [None req-3ccfa37e-4290-4b99-9849-91308c159545 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 24 05:57:46.251732 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:57:46.251732 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:57:46.251732 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:57:46.251732 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:57:46.251732 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:57:46.251732 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:57:46.251732 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:57:46.251732 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:57:46.251732 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:57:46.251732 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:57:46.251732 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:57:46.251732 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 05:57:46.251732 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 05:57:46.251732 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 05:57:46.251732 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 05:57:46.253078 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:57:46.253078 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:57:46.253078 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 05:57:46.253078 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 05:57:46.253078 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 05:57:46.253078 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 05:57:46.253078 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:57:46.253078 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:57:46.253078 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:57:46.253078 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:57:46.253078 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:57:46.253078 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application with Lock( Aug 24 05:57:46.253078 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:57:46.253078 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:57:46.253078 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:57:46.253078 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:57:46.253078 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:57:46.253078 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:57:46.254239 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:57:46.254239 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:57:46.254239 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 05:57:46.254239 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 05:57:46.254239 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 05:57:46.254239 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 05:57:46.254239 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 05:57:46.254239 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 05:57:46.254239 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 24 05:57:46.254239 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Aug 24 05:57:46.257726 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 301/595] 10.222.0.62 () {56 vars in 1102 bytes} [Wed Aug 24 05:57:46 2022] GET /identity/v3/users/neutron => generated 85 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:57:46.258904 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ece7ce-d7fb-4fae-bf3e-98644d489d37 None None] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:46.259187 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ece7ce-d7fb-4fae-bf3e-98644d489d37 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:46.259462 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ece7ce-d7fb-4fae-bf3e-98644d489d37 None None] PATH_INFO: `/` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:46.260874 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 302/596] 10.222.0.62 () {54 vars in 927 bytes} [Wed Aug 24 05:57:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:57:46.261493 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ece7ce-d7fb-4fae-bf3e-98644d489d37 None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:46.350235 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:46.353405 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:46.354535 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0643526-4760-4259-8477-fe4a28c0ee08 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:46.370494 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ece7ce-d7fb-4fae-bf3e-98644d489d37 None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:46.388891 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de4e944-2372-4f6f-a535-1713d4f2b61f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:46.389586 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de4e944-2372-4f6f-a535-1713d4f2b61f None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:46.389921 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de4e944-2372-4f6f-a535-1713d4f2b61f None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:46.390255 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de4e944-2372-4f6f-a535-1713d4f2b61f None admin] PATH_INFO: `/v3/domains` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:46.391460 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de4e944-2372-4f6f-a535-1713d4f2b61f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:57:46.391834 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de4e944-2372-4f6f-a535-1713d4f2b61f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:46.393281 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de4e944-2372-4f6f-a535-1713d4f2b61f None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:46.404763 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 295/597] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Aug 24 05:57:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:57:46.405655 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de4e944-2372-4f6f-a535-1713d4f2b61f None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:46.411685 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93557438-0d6d-4efc-9340-aec6c6c4308b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:46.412306 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93557438-0d6d-4efc-9340-aec6c6c4308b None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:46.412994 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93557438-0d6d-4efc-9340-aec6c6c4308b None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:46.413281 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93557438-0d6d-4efc-9340-aec6c6c4308b None admin] PATH_INFO: `/v3/projects` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:46.413960 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93557438-0d6d-4efc-9340-aec6c6c4308b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:57:46.414265 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93557438-0d6d-4efc-9340-aec6c6c4308b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:46.415706 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93557438-0d6d-4efc-9340-aec6c6c4308b None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:46.427983 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 303/598] 10.222.0.62 () {56 vars in 1148 bytes} [Wed Aug 24 05:57:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:57:46.429176 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7555d5e4-b834-44f0-9dbf-7ede49ed2d9f None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:46.429458 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7555d5e4-b834-44f0-9dbf-7ede49ed2d9f None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:46.429730 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7555d5e4-b834-44f0-9dbf-7ede49ed2d9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:46.447404 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-7555d5e4-b834-44f0-9dbf-7ede49ed2d9f None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:57:46.448599 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-7555d5e4-b834-44f0-9dbf-7ede49ed2d9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:57:46.519096 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 304/599] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Aug 24 05:57:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 05:57:46.520286 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7555d5e4-b834-44f0-9dbf-7ede49ed2d9f None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:46.533853 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de4e944-2372-4f6f-a535-1713d4f2b61f None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:46.579407 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88248153-6aab-4990-a76d-07e7ae6d5b4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:46.580557 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88248153-6aab-4990-a76d-07e7ae6d5b4a None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:46.581475 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88248153-6aab-4990-a76d-07e7ae6d5b4a None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:46.582243 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88248153-6aab-4990-a76d-07e7ae6d5b4a None admin] PATH_INFO: `/v3/users` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:46.583446 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88248153-6aab-4990-a76d-07e7ae6d5b4a None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:57:46.583954 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88248153-6aab-4990-a76d-07e7ae6d5b4a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:46.586263 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88248153-6aab-4990-a76d-07e7ae6d5b4a None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:46.598365 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:46.611216 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 296/600] 10.222.0.62 () {56 vars in 1139 bytes} [Wed Aug 24 05:57:46 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:57:46.612304 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88248153-6aab-4990-a76d-07e7ae6d5b4a None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:46.633989 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7555d5e4-b834-44f0-9dbf-7ede49ed2d9f None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:46.634676 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:46.670810 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab01266-a319-4f29-88e1-c4201db8e30d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:46.672783 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab01266-a319-4f29-88e1-c4201db8e30d None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:46.673166 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab01266-a319-4f29-88e1-c4201db8e30d None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:46.673512 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab01266-a319-4f29-88e1-c4201db8e30d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:46.683183 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab01266-a319-4f29-88e1-c4201db8e30d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:46.688103 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab01266-a319-4f29-88e1-c4201db8e30d None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:46.701815 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application [None req-0ab01266-a319-4f29-88e1-c4201db8e30d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 05:57:46.701815 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:57:46.701815 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:57:46.701815 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:57:46.701815 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:57:46.701815 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:57:46.701815 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:57:46.701815 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:57:46.701815 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:57:46.701815 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:57:46.701815 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:57:46.701815 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:57:46.701815 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 05:57:46.701815 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 05:57:46.701815 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 05:57:46.702943 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 05:57:46.702943 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:57:46.702943 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:57:46.702943 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:57:46.702943 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:57:46.702943 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:57:46.702943 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:57:46.702943 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:57:46.702943 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application with Lock( Aug 24 05:57:46.702943 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:57:46.702943 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:57:46.702943 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:57:46.702943 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:57:46.702943 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:57:46.702943 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:57:46.702943 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:57:46.702943 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:57:46.704023 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 05:57:46.704023 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 05:57:46.704023 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 05:57:46.704023 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 05:57:46.704023 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 05:57:46.704023 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 05:57:46.704023 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 05:57:46.704023 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Aug 24 05:57:46.713223 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 305/601] 10.222.0.62 () {56 vars in 1111 bytes} [Wed Aug 24 05:57:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:57:46.714256 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab01266-a319-4f29-88e1-c4201db8e30d None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:46.745902 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88248153-6aab-4990-a76d-07e7ae6d5b4a None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:46.784542 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d265dc-35da-4df1-b44e-b0f92ea5a690 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:46.785230 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d265dc-35da-4df1-b44e-b0f92ea5a690 None admin] REQUEST_METHOD: `PUT` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:46.785518 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d265dc-35da-4df1-b44e-b0f92ea5a690 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:46.785777 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d265dc-35da-4df1-b44e-b0f92ea5a690 None admin] PATH_INFO: `/v3/projects/181f4762037e454489d9552cc603179c/users/c7d52162cbdc4218b577ecc1e69a88c9/roles/5d95a5fc0f324a92905533aa8d34ac15` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:46.804722 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d265dc-35da-4df1-b44e-b0f92ea5a690 None admin] RBAC: Authorizing `identity:create_grant(project_id=181f4762037e454489d9552cc603179c, user_id=c7d52162cbdc4218b577ecc1e69a88c9, role_id=5d95a5fc0f324a92905533aa8d34ac15)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:46.806374 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d265dc-35da-4df1-b44e-b0f92ea5a690 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:46.830311 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab01266-a319-4f29-88e1-c4201db8e30d None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:46.866188 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da9af19-630e-43db-acfa-20a81989cd5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:46.867193 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da9af19-630e-43db-acfa-20a81989cd5d None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:46.867539 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da9af19-630e-43db-acfa-20a81989cd5d None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:46.867875 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da9af19-630e-43db-acfa-20a81989cd5d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:46.870502 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:46.873984 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 297/602] 10.222.0.62 () {58 vars in 1439 bytes} [Wed Aug 24 05:57:46 2022] PUT /identity/v3/projects/181f4762037e454489d9552cc603179c/users/c7d52162cbdc4218b577ecc1e69a88c9/roles/5d95a5fc0f324a92905533aa8d34ac15 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:57:46.879952 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da9af19-630e-43db-acfa-20a81989cd5d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:46.881589 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da9af19-630e-43db-acfa-20a81989cd5d None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:46.883270 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d265dc-35da-4df1-b44e-b0f92ea5a690 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:46.904820 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application [None req-4da9af19-630e-43db-acfa-20a81989cd5d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:57:46.904820 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:57:46.904820 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:57:46.904820 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:57:46.904820 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:57:46.904820 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:57:46.904820 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:57:46.904820 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:57:46.904820 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:57:46.904820 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:57:46.904820 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:57:46.904820 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:57:46.904820 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 05:57:46.904820 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 05:57:46.904820 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 05:57:46.904820 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 05:57:46.906678 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:57:46.906678 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:57:46.906678 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:57:46.906678 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:57:46.906678 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:57:46.906678 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:57:46.906678 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:57:46.906678 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application with Lock( Aug 24 05:57:46.906678 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:57:46.906678 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:57:46.906678 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:57:46.906678 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:57:46.906678 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:57:46.906678 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:57:46.906678 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:57:46.906678 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:57:46.906678 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 05:57:46.907796 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 05:57:46.907796 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:57:46.907796 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Aug 24 05:57:46.908427 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 306/603] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Aug 24 05:57:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:57:46.909591 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da9af19-630e-43db-acfa-20a81989cd5d None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:46.980702 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:47.030254 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d265dc-35da-4df1-b44e-b0f92ea5a690 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:47.048570 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da9af19-630e-43db-acfa-20a81989cd5d None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:47.066795 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bbfa8d-508b-41c0-a826-7f24f7735397 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:47.067404 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bbfa8d-508b-41c0-a826-7f24f7735397 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:47.067681 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bbfa8d-508b-41c0-a826-7f24f7735397 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:47.067951 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bbfa8d-508b-41c0-a826-7f24f7735397 None admin] PATH_INFO: `/v3/projects` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:47.068667 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bbfa8d-508b-41c0-a826-7f24f7735397 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:57:47.068970 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bbfa8d-508b-41c0-a826-7f24f7735397 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:47.070398 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bbfa8d-508b-41c0-a826-7f24f7735397 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:47.083639 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 298/604] 10.222.0.62 () {56 vars in 1148 bytes} [Wed Aug 24 05:57:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:57:47.088601 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bbfa8d-508b-41c0-a826-7f24f7735397 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:47.095241 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e1539c-f7b4-4192-8359-4515366f3f83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:47.096167 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e1539c-f7b4-4192-8359-4515366f3f83 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:47.096686 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e1539c-f7b4-4192-8359-4515366f3f83 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:47.097167 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e1539c-f7b4-4192-8359-4515366f3f83 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:47.108705 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e1539c-f7b4-4192-8359-4515366f3f83 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:57:47.114242 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application [None req-79e1539c-f7b4-4192-8359-4515366f3f83 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 05:57:47.114242 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:57:47.114242 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:57:47.114242 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:57:47.114242 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:57:47.114242 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:57:47.114242 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:57:47.114242 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:57:47.114242 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:57:47.114242 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:57:47.114242 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:57:47.114242 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:57:47.114242 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:57:47.114242 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:57:47.114242 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:57:47.114242 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:57:47.115177 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:57:47.115177 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:57:47.115177 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:57:47.115177 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:57:47.115177 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:57:47.115177 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:57:47.115177 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:57:47.115177 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:57:47.115177 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:57:47.115177 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:57:47.115177 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:57:47.115177 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application with Lock( Aug 24 05:57:47.115177 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:57:47.115177 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:57:47.115177 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:57:47.115177 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:57:47.116036 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:57:47.116036 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:57:47.116036 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:57:47.116036 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:57:47.116036 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:57:47.116036 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:57:47.116036 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:57:47.116036 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:57:47.116036 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:57:47.116036 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:57:47.116036 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 05:57:47.116036 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Aug 24 05:57:47.118739 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 307/605] 10.222.0.62 () {56 vars in 1102 bytes} [Wed Aug 24 05:57:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:57:47.119763 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e1539c-f7b4-4192-8359-4515366f3f83 None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:47.168893 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:47.191705 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bbfa8d-508b-41c0-a826-7f24f7735397 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:47.219322 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:47.228795 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e1539c-f7b4-4192-8359-4515366f3f83 None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:47.228956 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81543fc0-db40-4ea3-8bf0-485b8708b3a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:47.229515 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81543fc0-db40-4ea3-8bf0-485b8708b3a9 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:47.229786 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81543fc0-db40-4ea3-8bf0-485b8708b3a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:47.230059 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81543fc0-db40-4ea3-8bf0-485b8708b3a9 None admin] PATH_INFO: `/v3/domains` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:47.230696 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81543fc0-db40-4ea3-8bf0-485b8708b3a9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:57:47.230993 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81543fc0-db40-4ea3-8bf0-485b8708b3a9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:47.233187 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81543fc0-db40-4ea3-8bf0-485b8708b3a9 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:47.245171 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 299/606] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Aug 24 05:57:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:57:47.246009 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81543fc0-db40-4ea3-8bf0-485b8708b3a9 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:47.276816 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c14dd8e-78fc-44b0-ad0e-a8a0679be6fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:47.278473 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c14dd8e-78fc-44b0-ad0e-a8a0679be6fb None admin] REQUEST_METHOD: `PUT` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:47.279084 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c14dd8e-78fc-44b0-ad0e-a8a0679be6fb None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:47.279868 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c14dd8e-78fc-44b0-ad0e-a8a0679be6fb None admin] PATH_INFO: `/v3/projects/181f4762037e454489d9552cc603179c/users/8c4def55ac0e48b29836bc5c64bc7402/roles/5d95a5fc0f324a92905533aa8d34ac15` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:47.297601 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c14dd8e-78fc-44b0-ad0e-a8a0679be6fb None admin] RBAC: Authorizing `identity:create_grant(project_id=181f4762037e454489d9552cc603179c, user_id=8c4def55ac0e48b29836bc5c64bc7402, role_id=5d95a5fc0f324a92905533aa8d34ac15)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:47.299609 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c14dd8e-78fc-44b0-ad0e-a8a0679be6fb None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:47.352889 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81543fc0-db40-4ea3-8bf0-485b8708b3a9 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:47.380635 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 308/607] 10.222.0.62 () {58 vars in 1439 bytes} [Wed Aug 24 05:57:47 2022] PUT /identity/v3/projects/181f4762037e454489d9552cc603179c/users/8c4def55ac0e48b29836bc5c64bc7402/roles/5d95a5fc0f324a92905533aa8d34ac15 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:57:47.384629 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c14dd8e-78fc-44b0-ad0e-a8a0679be6fb None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:47.400739 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259e499a-9d6f-46dc-a354-8bdf2d041eb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:47.401371 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259e499a-9d6f-46dc-a354-8bdf2d041eb8 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:47.403604 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259e499a-9d6f-46dc-a354-8bdf2d041eb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:47.403961 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259e499a-9d6f-46dc-a354-8bdf2d041eb8 None admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:47.404691 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259e499a-9d6f-46dc-a354-8bdf2d041eb8 None admin] RBAC: Adding query filter params (name=service) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:57:47.405028 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259e499a-9d6f-46dc-a354-8bdf2d041eb8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:47.406801 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259e499a-9d6f-46dc-a354-8bdf2d041eb8 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:47.428119 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 300/608] 10.222.0.62 () {56 vars in 1103 bytes} [Wed Aug 24 05:57:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:57:47.435994 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259e499a-9d6f-46dc-a354-8bdf2d041eb8 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:47.547874 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:47.579346 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:47.587272 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c14dd8e-78fc-44b0-ad0e-a8a0679be6fb None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:47.611395 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259e499a-9d6f-46dc-a354-8bdf2d041eb8 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:47.629045 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a51a52-aaf7-44cf-9845-35b38fb43bd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:47.629678 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a51a52-aaf7-44cf-9845-35b38fb43bd1 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:47.629959 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a51a52-aaf7-44cf-9845-35b38fb43bd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:47.630232 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a51a52-aaf7-44cf-9845-35b38fb43bd1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:47.637208 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a51a52-aaf7-44cf-9845-35b38fb43bd1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:47.638666 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a51a52-aaf7-44cf-9845-35b38fb43bd1 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:47.648579 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25bd9a8-5234-4a2d-a120-339bfb85327d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:47.649218 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25bd9a8-5234-4a2d-a120-339bfb85327d None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:47.649567 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25bd9a8-5234-4a2d-a120-339bfb85327d None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:47.649845 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25bd9a8-5234-4a2d-a120-339bfb85327d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:47.652562 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application [None req-b7a51a52-aaf7-44cf-9845-35b38fb43bd1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 05:57:47.652562 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:57:47.652562 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:57:47.652562 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:57:47.652562 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:57:47.652562 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:57:47.652562 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:57:47.652562 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:57:47.652562 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:57:47.652562 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:57:47.652562 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:57:47.652562 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:57:47.652562 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 05:57:47.652562 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 05:57:47.652562 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 05:57:47.653419 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 05:57:47.653419 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:57:47.653419 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:57:47.653419 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:57:47.653419 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:57:47.653419 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:57:47.653419 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:57:47.653419 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:57:47.653419 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application with Lock( Aug 24 05:57:47.653419 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:57:47.653419 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:57:47.653419 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:57:47.653419 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:57:47.653419 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:57:47.653419 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:57:47.653419 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:57:47.653419 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:57:47.655275 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 05:57:47.655275 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 05:57:47.655275 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 05:57:47.655275 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 05:57:47.655275 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 05:57:47.655275 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 05:57:47.655275 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 05:57:47.655275 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Aug 24 05:57:47.655275 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 309/609] 10.222.0.62 () {56 vars in 1111 bytes} [Wed Aug 24 05:57:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:57:47.659762 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a51a52-aaf7-44cf-9845-35b38fb43bd1 None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:47.661506 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25bd9a8-5234-4a2d-a120-339bfb85327d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:47.663025 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25bd9a8-5234-4a2d-a120-339bfb85327d None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:47.678646 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-e25bd9a8-5234-4a2d-a120-339bfb85327d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:57:47.678646 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:57:47.678646 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:57:47.678646 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:57:47.678646 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:57:47.678646 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:57:47.678646 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:57:47.678646 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:57:47.678646 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:57:47.678646 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:57:47.678646 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:57:47.678646 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:57:47.678646 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 05:57:47.678646 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 05:57:47.678646 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 05:57:47.678646 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 05:57:47.680715 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:57:47.680715 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:57:47.680715 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:57:47.680715 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:57:47.680715 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:57:47.680715 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:57:47.680715 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:57:47.680715 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 24 05:57:47.680715 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:57:47.680715 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:57:47.680715 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:57:47.680715 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:57:47.680715 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:57:47.680715 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:57:47.680715 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:57:47.680715 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:57:47.680715 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 05:57:47.681689 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 05:57:47.681689 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:57:47.681689 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 24 05:57:47.681689 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 301/610] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Aug 24 05:57:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:57:47.693934 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25bd9a8-5234-4a2d-a120-339bfb85327d None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:47.771408 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a51a52-aaf7-44cf-9845-35b38fb43bd1 None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:47.807310 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b830ca-0fd1-4ae1-9a25-eea2c5583c11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:47.808056 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b830ca-0fd1-4ae1-9a25-eea2c5583c11 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:47.808339 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b830ca-0fd1-4ae1-9a25-eea2c5583c11 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:47.808633 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b830ca-0fd1-4ae1-9a25-eea2c5583c11 None admin] PATH_INFO: `/v3/projects` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:47.809585 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b830ca-0fd1-4ae1-9a25-eea2c5583c11 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:57:47.810317 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b830ca-0fd1-4ae1-9a25-eea2c5583c11 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:47.811771 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b830ca-0fd1-4ae1-9a25-eea2c5583c11 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:47.827746 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 310/611] 10.222.0.62 () {56 vars in 1148 bytes} [Wed Aug 24 05:57:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:57:47.833885 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b830ca-0fd1-4ae1-9a25-eea2c5583c11 None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:47.864931 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:47.869212 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:47.873093 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25bd9a8-5234-4a2d-a120-339bfb85327d None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:47.908728 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335f3f71-4a2c-4dbb-be3f-fbd9c0fb6b7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:47.909182 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335f3f71-4a2c-4dbb-be3f-fbd9c0fb6b7e None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:47.909461 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335f3f71-4a2c-4dbb-be3f-fbd9c0fb6b7e None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:47.909732 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335f3f71-4a2c-4dbb-be3f-fbd9c0fb6b7e None admin] PATH_INFO: `/v3/domains` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:47.910465 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335f3f71-4a2c-4dbb-be3f-fbd9c0fb6b7e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:57:47.910720 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335f3f71-4a2c-4dbb-be3f-fbd9c0fb6b7e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:47.912144 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335f3f71-4a2c-4dbb-be3f-fbd9c0fb6b7e None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:47.929970 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 302/612] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Aug 24 05:57:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:57:47.931336 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b91c19-1d74-4113-8d14-b632180e9be5 None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:47.931682 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b91c19-1d74-4113-8d14-b632180e9be5 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:47.931960 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b91c19-1d74-4113-8d14-b632180e9be5 None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:47.933547 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 303/613] 10.222.0.62 () {54 vars in 927 bytes} [Wed Aug 24 05:57:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:57:47.934269 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b91c19-1d74-4113-8d14-b632180e9be5 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:47.954185 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b830ca-0fd1-4ae1-9a25-eea2c5583c11 None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:47.993043 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37abb2b9-f229-4950-9e4e-b24f1050ff65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:47.994080 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37abb2b9-f229-4950-9e4e-b24f1050ff65 None admin] REQUEST_METHOD: `PUT` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:47.994556 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37abb2b9-f229-4950-9e4e-b24f1050ff65 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:47.995020 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37abb2b9-f229-4950-9e4e-b24f1050ff65 None admin] PATH_INFO: `/v3/projects/181f4762037e454489d9552cc603179c/users/0bb6c79db75e4d169331c02133d1ec32/roles/5d95a5fc0f324a92905533aa8d34ac15` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:48.011008 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37abb2b9-f229-4950-9e4e-b24f1050ff65 None admin] RBAC: Authorizing `identity:create_grant(project_id=181f4762037e454489d9552cc603179c, user_id=0bb6c79db75e4d169331c02133d1ec32, role_id=5d95a5fc0f324a92905533aa8d34ac15)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:48.012748 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37abb2b9-f229-4950-9e4e-b24f1050ff65 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:48.051152 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b91c19-1d74-4113-8d14-b632180e9be5 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:48.085799 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683e0531-b228-43d0-8b4c-51cf9734319b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:48.086350 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683e0531-b228-43d0-8b4c-51cf9734319b None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:48.086628 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683e0531-b228-43d0-8b4c-51cf9734319b None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:48.086901 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683e0531-b228-43d0-8b4c-51cf9734319b None admin] PATH_INFO: `/v3/users/swift` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:48.100746 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 311/614] 10.222.0.62 () {58 vars in 1439 bytes} [Wed Aug 24 05:57:47 2022] PUT /identity/v3/projects/181f4762037e454489d9552cc603179c/users/0bb6c79db75e4d169331c02133d1ec32/roles/5d95a5fc0f324a92905533aa8d34ac15 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:57:48.103294 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683e0531-b228-43d0-8b4c-51cf9734319b None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:48.105017 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1145ed6b-ebb3-48fb-86f5-e71cb936ff20 None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:48.105143 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1145ed6b-ebb3-48fb-86f5-e71cb936ff20 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:48.105507 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1145ed6b-ebb3-48fb-86f5-e71cb936ff20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:48.106319 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683e0531-b228-43d0-8b4c-51cf9734319b None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:48.132423 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-683e0531-b228-43d0-8b4c-51cf9734319b None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 24 05:57:48.132423 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:57:48.132423 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:57:48.132423 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:57:48.132423 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:57:48.132423 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:57:48.132423 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:57:48.132423 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:57:48.132423 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:57:48.132423 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:57:48.132423 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:57:48.132423 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:57:48.132423 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 05:57:48.132423 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 05:57:48.132423 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 05:57:48.132423 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 05:57:48.133495 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:57:48.133495 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:57:48.133495 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 05:57:48.133495 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 05:57:48.133495 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 05:57:48.133495 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 05:57:48.133495 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:57:48.133495 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:57:48.133495 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:57:48.133495 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:57:48.133495 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:57:48.133495 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 24 05:57:48.133495 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:57:48.133495 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:57:48.133495 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:57:48.133495 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:57:48.133495 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:57:48.133495 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:57:48.134459 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:57:48.134459 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:57:48.134459 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 05:57:48.134459 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 05:57:48.134459 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 05:57:48.134459 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 05:57:48.134459 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 05:57:48.134459 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 05:57:48.134459 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 24 05:57:48.134459 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 24 05:57:48.134979 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 304/615] 10.222.0.62 () {56 vars in 1096 bytes} [Wed Aug 24 05:57:47 2022] GET /identity/v3/users/swift => generated 83 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:57:48.141902 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683e0531-b228-43d0-8b4c-51cf9734319b None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:48.148825 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-1145ed6b-ebb3-48fb-86f5-e71cb936ff20 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:57:48.151169 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-1145ed6b-ebb3-48fb-86f5-e71cb936ff20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:57:48.188422 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:48.266875 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:48.284659 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 312/616] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Aug 24 05:57:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 05:57:48.315424 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683e0531-b228-43d0-8b4c-51cf9734319b None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:48.352329 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b0fd1f-9eb0-44e6-b090-088fe6b192f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:48.352943 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b0fd1f-9eb0-44e6-b090-088fe6b192f9 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:48.353476 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b0fd1f-9eb0-44e6-b090-088fe6b192f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:48.353476 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b0fd1f-9eb0-44e6-b090-088fe6b192f9 None admin] PATH_INFO: `/v3/users` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:48.354286 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b0fd1f-9eb0-44e6-b090-088fe6b192f9 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:57:48.354595 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b0fd1f-9eb0-44e6-b090-088fe6b192f9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:48.356023 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b0fd1f-9eb0-44e6-b090-088fe6b192f9 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:48.374145 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 305/617] 10.222.0.62 () {56 vars in 1135 bytes} [Wed Aug 24 05:57:48 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:57:48.379398 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1145ed6b-ebb3-48fb-86f5-e71cb936ff20 None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:48.492712 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1145ed6b-ebb3-48fb-86f5-e71cb936ff20 None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:48.528521 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e85e8f1-129f-4094-98c4-48b66dd9fefe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:48.529151 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e85e8f1-129f-4094-98c4-48b66dd9fefe None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:48.529434 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e85e8f1-129f-4094-98c4-48b66dd9fefe None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:48.529705 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e85e8f1-129f-4094-98c4-48b66dd9fefe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:48.540329 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e85e8f1-129f-4094-98c4-48b66dd9fefe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:48.541888 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e85e8f1-129f-4094-98c4-48b66dd9fefe None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:48.557708 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application [None req-0e85e8f1-129f-4094-98c4-48b66dd9fefe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:57:48.557708 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:57:48.557708 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:57:48.557708 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:57:48.557708 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:57:48.557708 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:57:48.557708 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:57:48.557708 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:57:48.557708 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:57:48.557708 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:57:48.557708 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:57:48.557708 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:57:48.557708 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 05:57:48.557708 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 05:57:48.557708 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 05:57:48.557708 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 05:57:48.558643 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:57:48.558643 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:57:48.558643 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:57:48.558643 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:57:48.558643 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:57:48.558643 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:57:48.558643 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:57:48.558643 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application with Lock( Aug 24 05:57:48.558643 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:57:48.558643 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:57:48.558643 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:57:48.558643 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:57:48.558643 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:57:48.558643 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:57:48.558643 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:57:48.558643 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:57:48.558643 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 05:57:48.559524 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 05:57:48.559524 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:57:48.559524 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Aug 24 05:57:48.559524 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 313/618] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Aug 24 05:57:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:57:48.565559 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b0fd1f-9eb0-44e6-b090-088fe6b192f9 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:48.685093 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e85e8f1-129f-4094-98c4-48b66dd9fefe None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:48.688993 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:48.738149 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b0fd1f-9eb0-44e6-b090-088fe6b192f9 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:48.780921 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:48.812702 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56804f0e-8988-4e63-870d-8558ef657de3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:48.813489 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56804f0e-8988-4e63-870d-8558ef657de3 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:48.813780 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56804f0e-8988-4e63-870d-8558ef657de3 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:48.814055 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56804f0e-8988-4e63-870d-8558ef657de3 None admin] PATH_INFO: `/v3/domains` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:48.814707 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56804f0e-8988-4e63-870d-8558ef657de3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:57:48.815015 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56804f0e-8988-4e63-870d-8558ef657de3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:48.816393 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56804f0e-8988-4e63-870d-8558ef657de3 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:48.836917 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 306/619] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Aug 24 05:57:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:57:48.837469 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e85e8f1-129f-4094-98c4-48b66dd9fefe None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:48.844594 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56804f0e-8988-4e63-870d-8558ef657de3 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:48.874584 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de31f19-5836-4f39-a263-1f89effee2d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:48.875327 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de31f19-5836-4f39-a263-1f89effee2d2 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:48.875647 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de31f19-5836-4f39-a263-1f89effee2d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:48.875994 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de31f19-5836-4f39-a263-1f89effee2d2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:48.886367 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de31f19-5836-4f39-a263-1f89effee2d2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:57:48.891601 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application [None req-8de31f19-5836-4f39-a263-1f89effee2d2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 05:57:48.891601 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:57:48.891601 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:57:48.891601 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:57:48.891601 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:57:48.891601 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:57:48.891601 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:57:48.891601 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:57:48.891601 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:57:48.891601 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:57:48.891601 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:57:48.891601 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:57:48.891601 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:57:48.891601 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:57:48.891601 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:57:48.891601 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:57:48.892560 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:57:48.892560 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:57:48.892560 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:57:48.892560 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:57:48.892560 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:57:48.892560 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:57:48.892560 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:57:48.892560 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:57:48.892560 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:57:48.892560 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:57:48.892560 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:57:48.892560 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application with Lock( Aug 24 05:57:48.892560 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:57:48.892560 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:57:48.892560 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:57:48.892560 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:57:48.893625 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:57:48.893625 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:57:48.893625 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:57:48.893625 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:57:48.893625 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:57:48.893625 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:57:48.893625 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:57:48.893625 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:57:48.893625 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:57:48.893625 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:57:48.893625 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 05:57:48.893625 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Aug 24 05:57:48.893625 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 314/620] 10.222.0.62 () {56 vars in 1102 bytes} [Wed Aug 24 05:57:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 05:57:48.906337 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8538b8ed-a0ce-4e90-8c0f-534d998496b6 None None] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:48.908532 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8538b8ed-a0ce-4e90-8c0f-534d998496b6 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:48.908532 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8538b8ed-a0ce-4e90-8c0f-534d998496b6 None None] PATH_INFO: `/` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:48.908532 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 315/621] 10.222.0.62 () {54 vars in 927 bytes} [Wed Aug 24 05:57:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:57:48.909129 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8538b8ed-a0ce-4e90-8c0f-534d998496b6 None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:48.971502 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56804f0e-8988-4e63-870d-8558ef657de3 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:49.008293 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235cd2ed-ce3f-4c34-ab26-49c066e7577e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:49.008986 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235cd2ed-ce3f-4c34-ab26-49c066e7577e None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:49.009268 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235cd2ed-ce3f-4c34-ab26-49c066e7577e None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:49.009545 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235cd2ed-ce3f-4c34-ab26-49c066e7577e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:49.013542 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8538b8ed-a0ce-4e90-8c0f-534d998496b6 None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:49.017373 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235cd2ed-ce3f-4c34-ab26-49c066e7577e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:49.019011 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235cd2ed-ce3f-4c34-ab26-49c066e7577e None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:49.022416 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:49.026129 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:49.034110 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-235cd2ed-ce3f-4c34-ab26-49c066e7577e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 05:57:49.034110 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:57:49.034110 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:57:49.034110 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:57:49.034110 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:57:49.034110 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:57:49.034110 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:57:49.034110 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:57:49.034110 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:57:49.034110 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:57:49.034110 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:57:49.034110 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:57:49.034110 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 05:57:49.034110 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 05:57:49.034110 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 05:57:49.035223 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 05:57:49.035223 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:57:49.035223 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:57:49.035223 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:57:49.035223 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:57:49.035223 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:57:49.035223 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:57:49.035223 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:57:49.035223 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 24 05:57:49.035223 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:57:49.035223 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:57:49.035223 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:57:49.035223 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:57:49.035223 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:57:49.035223 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:57:49.035223 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:57:49.035223 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:57:49.036157 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 05:57:49.036157 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 05:57:49.036157 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 05:57:49.036157 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 05:57:49.036157 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 05:57:49.036157 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 05:57:49.036157 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 05:57:49.036157 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 24 05:57:49.040385 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 307/622] 10.222.0.62 () {56 vars in 1111 bytes} [Wed Aug 24 05:57:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:57:49.042081 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7246b70e-136a-4a55-a0de-68fc5c8ca5e5 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:49.056049 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7246b70e-136a-4a55-a0de-68fc5c8ca5e5 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:49.056049 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7246b70e-136a-4a55-a0de-68fc5c8ca5e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:49.056279 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046181f7-ecc6-40e9-999b-54d4d4e56ef4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:49.056279 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046181f7-ecc6-40e9-999b-54d4d4e56ef4 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:49.056683 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046181f7-ecc6-40e9-999b-54d4d4e56ef4 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:49.057110 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046181f7-ecc6-40e9-999b-54d4d4e56ef4 None admin] PATH_INFO: `/v3/roles` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:49.057905 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046181f7-ecc6-40e9-999b-54d4d4e56ef4 None admin] RBAC: Adding query filter params (name=service) {{(pid=81851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:57:49.058346 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046181f7-ecc6-40e9-999b-54d4d4e56ef4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:49.059878 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046181f7-ecc6-40e9-999b-54d4d4e56ef4 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:49.065581 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-7246b70e-136a-4a55-a0de-68fc5c8ca5e5 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:57:49.067420 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-7246b70e-136a-4a55-a0de-68fc5c8ca5e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:57:49.069838 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 316/623] 10.222.0.62 () {56 vars in 1103 bytes} [Wed Aug 24 05:57:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:57:49.075468 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046181f7-ecc6-40e9-999b-54d4d4e56ef4 None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:49.150259 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 308/624] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Aug 24 05:57:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 05:57:49.152084 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7246b70e-136a-4a55-a0de-68fc5c8ca5e5 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:49.201304 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046181f7-ecc6-40e9-999b-54d4d4e56ef4 None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:49.237067 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22507306-603d-4c76-8540-ded9b22c02c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:49.237887 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22507306-603d-4c76-8540-ded9b22c02c0 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:49.238310 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22507306-603d-4c76-8540-ded9b22c02c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:49.238721 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22507306-603d-4c76-8540-ded9b22c02c0 None admin] PATH_INFO: `/v3/projects` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:49.239562 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22507306-603d-4c76-8540-ded9b22c02c0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:57:49.240007 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22507306-603d-4c76-8540-ded9b22c02c0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:49.241611 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22507306-603d-4c76-8540-ded9b22c02c0 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:49.255355 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 317/625] 10.222.0.62 () {56 vars in 1148 bytes} [Wed Aug 24 05:57:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:57:49.261951 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22507306-603d-4c76-8540-ded9b22c02c0 None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:49.317805 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:49.388904 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7246b70e-136a-4a55-a0de-68fc5c8ca5e5 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:49.420707 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22507306-603d-4c76-8540-ded9b22c02c0 None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:49.439319 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7bb1e8-b491-4de3-b507-b2d8ac8ee3db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:49.440002 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7bb1e8-b491-4de3-b507-b2d8ac8ee3db None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:49.440296 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7bb1e8-b491-4de3-b507-b2d8ac8ee3db None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:49.440747 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7bb1e8-b491-4de3-b507-b2d8ac8ee3db None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:49.450952 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7bb1e8-b491-4de3-b507-b2d8ac8ee3db None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:49.452489 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7bb1e8-b491-4de3-b507-b2d8ac8ee3db None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:49.458370 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d60cea-89ec-40c5-9232-ccb7cc2e781c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:49.459531 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d60cea-89ec-40c5-9232-ccb7cc2e781c None admin] REQUEST_METHOD: `PUT` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:49.459711 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:49.460125 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d60cea-89ec-40c5-9232-ccb7cc2e781c None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:49.460561 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d60cea-89ec-40c5-9232-ccb7cc2e781c None admin] PATH_INFO: `/v3/projects/181f4762037e454489d9552cc603179c/users/14d4fed04fb144d08918bd5d4f808d23/roles/5d95a5fc0f324a92905533aa8d34ac15` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:49.471480 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-3f7bb1e8-b491-4de3-b507-b2d8ac8ee3db None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:57:49.471480 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:57:49.471480 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:57:49.471480 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:57:49.471480 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:57:49.471480 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:57:49.471480 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:57:49.471480 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:57:49.471480 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:57:49.471480 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:57:49.471480 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:57:49.471480 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:57:49.471480 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 05:57:49.471480 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 05:57:49.471480 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 05:57:49.471480 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 05:57:49.479410 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:57:49.479410 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:57:49.479410 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:57:49.479410 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:57:49.479410 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:57:49.479410 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:57:49.479410 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:57:49.479410 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 24 05:57:49.479410 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:57:49.479410 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:57:49.479410 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:57:49.479410 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:57:49.479410 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:57:49.479410 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:57:49.479410 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:57:49.479410 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:57:49.479410 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 05:57:49.480310 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 05:57:49.480310 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:57:49.480310 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 24 05:57:49.480310 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 309/626] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Aug 24 05:57:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 321 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:57:49.480310 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7bb1e8-b491-4de3-b507-b2d8ac8ee3db None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:49.486480 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d60cea-89ec-40c5-9232-ccb7cc2e781c None admin] RBAC: Authorizing `identity:create_grant(project_id=181f4762037e454489d9552cc603179c, user_id=14d4fed04fb144d08918bd5d4f808d23, role_id=5d95a5fc0f324a92905533aa8d34ac15)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:49.488883 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d60cea-89ec-40c5-9232-ccb7cc2e781c None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:49.573397 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 318/627] 10.222.0.62 () {58 vars in 1439 bytes} [Wed Aug 24 05:57:49 2022] PUT /identity/v3/projects/181f4762037e454489d9552cc603179c/users/14d4fed04fb144d08918bd5d4f808d23/roles/5d95a5fc0f324a92905533aa8d34ac15 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 05:57:49.577774 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d60cea-89ec-40c5-9232-ccb7cc2e781c None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:49.606008 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7bb1e8-b491-4de3-b507-b2d8ac8ee3db None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:49.666526 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8a1610-6058-42ff-aac8-56220779a116 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:49.667173 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8a1610-6058-42ff-aac8-56220779a116 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:49.667457 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8a1610-6058-42ff-aac8-56220779a116 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:49.667732 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8a1610-6058-42ff-aac8-56220779a116 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:49.679999 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8a1610-6058-42ff-aac8-56220779a116 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:57:49.690219 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-9b8a1610-6058-42ff-aac8-56220779a116 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 05:57:49.690219 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:57:49.690219 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:57:49.690219 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:57:49.690219 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:57:49.690219 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:57:49.690219 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:57:49.690219 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:57:49.690219 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:57:49.690219 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:57:49.690219 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:57:49.690219 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:57:49.690219 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:57:49.690219 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:57:49.690219 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:57:49.690219 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:57:49.691282 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:57:49.691282 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:57:49.691282 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:57:49.691282 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:57:49.691282 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:57:49.691282 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:57:49.691282 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:57:49.691282 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:57:49.691282 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:57:49.691282 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:57:49.691282 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:57:49.691282 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 24 05:57:49.691282 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:57:49.691282 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:57:49.691282 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:57:49.691282 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:57:49.692162 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:57:49.692162 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:57:49.692162 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:57:49.692162 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:57:49.692162 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:57:49.692162 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:57:49.692162 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:57:49.692162 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:57:49.692162 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:57:49.692162 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:57:49.692162 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 05:57:49.692162 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 24 05:57:49.692162 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 310/628] 10.222.0.62 () {56 vars in 1102 bytes} [Wed Aug 24 05:57:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:57:49.692977 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198c9953-4f4d-47df-8a23-beafa65659eb None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:49.692977 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198c9953-4f4d-47df-8a23-beafa65659eb None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:49.693612 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198c9953-4f4d-47df-8a23-beafa65659eb None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:49.695103 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 311/629] 10.222.0.62 () {54 vars in 927 bytes} [Wed Aug 24 05:57:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:57:49.700527 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198c9953-4f4d-47df-8a23-beafa65659eb None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:49.723697 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:49.755038 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d60cea-89ec-40c5-9232-ccb7cc2e781c None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:49.763266 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:49.804046 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5ee2fd-0782-443c-ad22-2b74a4162e05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:49.805304 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5ee2fd-0782-443c-ad22-2b74a4162e05 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:49.805304 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5ee2fd-0782-443c-ad22-2b74a4162e05 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:49.805304 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5ee2fd-0782-443c-ad22-2b74a4162e05 None admin] PATH_INFO: `/v3/domains` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:49.805995 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5ee2fd-0782-443c-ad22-2b74a4162e05 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:57:49.806302 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5ee2fd-0782-443c-ad22-2b74a4162e05 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:49.807685 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5ee2fd-0782-443c-ad22-2b74a4162e05 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:49.820556 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 319/630] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Aug 24 05:57:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:57:49.826320 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f636a7f-c027-4570-8bee-2661abd18ed5 None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:49.826320 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f636a7f-c027-4570-8bee-2661abd18ed5 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:49.826320 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f636a7f-c027-4570-8bee-2661abd18ed5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:49.840903 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198c9953-4f4d-47df-8a23-beafa65659eb None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:49.841333 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-9f636a7f-c027-4570-8bee-2661abd18ed5 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:57:49.842565 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-9f636a7f-c027-4570-8bee-2661abd18ed5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:57:49.888042 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8f36c8-d4e1-4eb4-9c11-de62dab697cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:49.889071 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8f36c8-d4e1-4eb4-9c11-de62dab697cb None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:49.889554 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8f36c8-d4e1-4eb4-9c11-de62dab697cb None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:49.890026 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8f36c8-d4e1-4eb4-9c11-de62dab697cb None admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:49.891404 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8f36c8-d4e1-4eb4-9c11-de62dab697cb None admin] RBAC: Adding query filter params (name=service) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:57:49.891923 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8f36c8-d4e1-4eb4-9c11-de62dab697cb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:49.894002 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8f36c8-d4e1-4eb4-9c11-de62dab697cb None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:49.905576 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 312/631] 10.222.0.62 () {56 vars in 1103 bytes} [Wed Aug 24 05:57:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:57:49.907023 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8f36c8-d4e1-4eb4-9c11-de62dab697cb None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:49.926081 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 320/632] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Aug 24 05:57:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 05:57:49.927506 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f636a7f-c027-4570-8bee-2661abd18ed5 None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:50.019219 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8f36c8-d4e1-4eb4-9c11-de62dab697cb None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:50.020648 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:50.056024 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a197eed-0f05-4154-b12d-be77265fc26c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:50.056701 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a197eed-0f05-4154-b12d-be77265fc26c None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:50.057032 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a197eed-0f05-4154-b12d-be77265fc26c None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:50.057310 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a197eed-0f05-4154-b12d-be77265fc26c None admin] PATH_INFO: `/v3/users/nova` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:50.066697 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a197eed-0f05-4154-b12d-be77265fc26c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:50.068136 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a197eed-0f05-4154-b12d-be77265fc26c None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:50.088362 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-4a197eed-0f05-4154-b12d-be77265fc26c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 05:57:50.088362 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:57:50.088362 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:57:50.088362 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:57:50.088362 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:57:50.088362 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:57:50.088362 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:57:50.088362 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:57:50.088362 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:57:50.088362 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:57:50.088362 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:57:50.088362 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:57:50.088362 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 05:57:50.088362 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 05:57:50.088362 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 05:57:50.088362 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 05:57:50.089384 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:57:50.089384 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:57:50.089384 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 05:57:50.089384 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 05:57:50.089384 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 05:57:50.089384 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 05:57:50.089384 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:57:50.089384 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:57:50.089384 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:57:50.089384 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:57:50.089384 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:57:50.089384 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 24 05:57:50.089384 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:57:50.089384 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:57:50.089384 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:57:50.089384 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:57:50.089384 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:57:50.089384 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:57:50.090318 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:57:50.090318 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:57:50.090318 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 05:57:50.090318 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 05:57:50.090318 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 05:57:50.090318 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 05:57:50.090318 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 05:57:50.090318 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 05:57:50.090318 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 05:57:50.090318 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 24 05:57:50.090854 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 313/633] 10.222.0.62 () {56 vars in 1093 bytes} [Wed Aug 24 05:57:49 2022] GET /identity/v3/users/nova => generated 82 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:57:50.102983 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a197eed-0f05-4154-b12d-be77265fc26c None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:50.105394 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:50.151272 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f636a7f-c027-4570-8bee-2661abd18ed5 None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:50.211079 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a197eed-0f05-4154-b12d-be77265fc26c None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:50.221895 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618c6c78-2e72-4c18-911b-959591a8d6c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:50.222544 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618c6c78-2e72-4c18-911b-959591a8d6c3 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:50.223231 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618c6c78-2e72-4c18-911b-959591a8d6c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:50.223510 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618c6c78-2e72-4c18-911b-959591a8d6c3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:50.237987 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618c6c78-2e72-4c18-911b-959591a8d6c3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:50.239439 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618c6c78-2e72-4c18-911b-959591a8d6c3 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:50.256762 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859b68f0-49fe-4fb2-8809-fa24af89b3cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:50.257204 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application [None req-618c6c78-2e72-4c18-911b-959591a8d6c3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:57:50.257204 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:57:50.257204 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:57:50.257204 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:57:50.257204 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:57:50.257204 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:57:50.257204 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:57:50.257204 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:57:50.257204 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:57:50.257204 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:57:50.257204 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:57:50.257204 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:57:50.257204 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 05:57:50.257204 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 05:57:50.257204 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 05:57:50.257204 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 05:57:50.258109 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:57:50.258109 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:57:50.258109 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:57:50.258109 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:57:50.258109 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:57:50.258109 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:57:50.258109 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:57:50.258109 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application with Lock( Aug 24 05:57:50.258109 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:57:50.258109 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:57:50.258109 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:57:50.258109 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:57:50.258109 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:57:50.258109 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:57:50.258109 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:57:50.258109 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:57:50.258109 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 05:57:50.258994 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 05:57:50.258994 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:57:50.258994 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Aug 24 05:57:50.258994 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 321/634] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Aug 24 05:57:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 332 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:57:50.259644 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618c6c78-2e72-4c18-911b-959591a8d6c3 None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:50.260408 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859b68f0-49fe-4fb2-8809-fa24af89b3cd None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:50.260930 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859b68f0-49fe-4fb2-8809-fa24af89b3cd None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:50.261430 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859b68f0-49fe-4fb2-8809-fa24af89b3cd None admin] PATH_INFO: `/v3/users` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:50.262822 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859b68f0-49fe-4fb2-8809-fa24af89b3cd None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:57:50.264316 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859b68f0-49fe-4fb2-8809-fa24af89b3cd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:50.266585 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859b68f0-49fe-4fb2-8809-fa24af89b3cd None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:50.295297 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 314/635] 10.222.0.62 () {56 vars in 1133 bytes} [Wed Aug 24 05:57:50 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:57:50.295297 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859b68f0-49fe-4fb2-8809-fa24af89b3cd None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:50.325005 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:50.386634 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618c6c78-2e72-4c18-911b-959591a8d6c3 None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:50.413756 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:50.416767 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859b68f0-49fe-4fb2-8809-fa24af89b3cd None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:50.433057 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc23f13-e740-42a9-84d2-ec7d68d0e7bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:50.433968 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc23f13-e740-42a9-84d2-ec7d68d0e7bb None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:50.434248 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc23f13-e740-42a9-84d2-ec7d68d0e7bb None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:50.434519 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc23f13-e740-42a9-84d2-ec7d68d0e7bb None admin] PATH_INFO: `/v3/roles/service` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:50.448916 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc23f13-e740-42a9-84d2-ec7d68d0e7bb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:57:50.452584 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b3de39-6a18-4812-84fc-8e29305a8c2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:50.453791 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b3de39-6a18-4812-84fc-8e29305a8c2c None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:50.454068 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b3de39-6a18-4812-84fc-8e29305a8c2c None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:50.454335 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b3de39-6a18-4812-84fc-8e29305a8c2c None admin] PATH_INFO: `/v3/domains` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:50.454800 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application [None req-2cc23f13-e740-42a9-84d2-ec7d68d0e7bb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 05:57:50.454800 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:57:50.454800 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:57:50.454800 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:57:50.454800 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:57:50.454800 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:57:50.454800 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:57:50.454800 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:57:50.454800 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:57:50.454800 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:57:50.454800 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:57:50.454800 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:57:50.454800 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:57:50.454800 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:57:50.454800 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:57:50.454800 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:57:50.455713 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:57:50.455713 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:57:50.455713 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:57:50.455713 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:57:50.455713 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:57:50.455713 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:57:50.455713 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:57:50.455713 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:57:50.455713 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:57:50.455713 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:57:50.455713 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:57:50.455713 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application with Lock( Aug 24 05:57:50.455713 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:57:50.455713 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:57:50.455713 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:57:50.455713 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:57:50.456561 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:57:50.456561 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:57:50.456561 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:57:50.456561 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:57:50.456561 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:57:50.456561 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:57:50.456561 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:57:50.456561 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:57:50.456561 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:57:50.456561 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:57:50.456561 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 05:57:50.456561 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Aug 24 05:57:50.457207 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b3de39-6a18-4812-84fc-8e29305a8c2c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:57:50.457207 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b3de39-6a18-4812-84fc-8e29305a8c2c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:50.457897 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 322/636] 10.222.0.62 () {56 vars in 1102 bytes} [Wed Aug 24 05:57:50 2022] GET /identity/v3/roles/service => generated 85 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:57:50.458002 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b3de39-6a18-4812-84fc-8e29305a8c2c None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:50.458168 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc23f13-e740-42a9-84d2-ec7d68d0e7bb None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:50.474003 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 315/637] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Aug 24 05:57:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:57:50.474003 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c45629-b883-4f8b-a247-b49eab09f2ff None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:50.474003 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c45629-b883-4f8b-a247-b49eab09f2ff None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:50.474003 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c45629-b883-4f8b-a247-b49eab09f2ff None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:50.474003 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 316/638] 10.222.0.62 () {54 vars in 927 bytes} [Wed Aug 24 05:57:50 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:57:50.474003 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c45629-b883-4f8b-a247-b49eab09f2ff None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:50.559546 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:50.568075 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc23f13-e740-42a9-84d2-ec7d68d0e7bb None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:50.580155 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c45629-b883-4f8b-a247-b49eab09f2ff None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:50.600971 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609f0e4e-b165-4a42-b302-b13a47b35d3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:50.601598 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609f0e4e-b165-4a42-b302-b13a47b35d3a None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:50.601873 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609f0e4e-b165-4a42-b302-b13a47b35d3a None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:50.602142 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609f0e4e-b165-4a42-b302-b13a47b35d3a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:50.616649 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c854f86-9f77-4fc8-98ce-cb0c5eb57cef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:50.617613 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c854f86-9f77-4fc8-98ce-cb0c5eb57cef None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:50.618168 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c854f86-9f77-4fc8-98ce-cb0c5eb57cef None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:50.618588 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c854f86-9f77-4fc8-98ce-cb0c5eb57cef None admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:50.619530 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c854f86-9f77-4fc8-98ce-cb0c5eb57cef None admin] RBAC: Adding query filter params (name=service) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:57:50.619982 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c854f86-9f77-4fc8-98ce-cb0c5eb57cef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:50.621624 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c854f86-9f77-4fc8-98ce-cb0c5eb57cef None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:50.635975 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 317/639] 10.222.0.62 () {56 vars in 1103 bytes} [Wed Aug 24 05:57:50 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:57:50.637060 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c854f86-9f77-4fc8-98ce-cb0c5eb57cef None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:50.651478 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:50.670917 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609f0e4e-b165-4a42-b302-b13a47b35d3a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:50.673111 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609f0e4e-b165-4a42-b302-b13a47b35d3a None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:50.740014 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c854f86-9f77-4fc8-98ce-cb0c5eb57cef None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:50.778073 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f8ffb8-08bf-40aa-b875-768b9649750a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:50.778682 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f8ffb8-08bf-40aa-b875-768b9649750a None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:50.778961 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f8ffb8-08bf-40aa-b875-768b9649750a None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:50.779239 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f8ffb8-08bf-40aa-b875-768b9649750a None admin] PATH_INFO: `/v3/users/glance` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:50.791887 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 323/640] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Aug 24 05:57:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 05:57:50.791887 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901a7e09-41f1-4803-88b5-98b76097ce10 None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:50.791887 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901a7e09-41f1-4803-88b5-98b76097ce10 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:50.791887 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901a7e09-41f1-4803-88b5-98b76097ce10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:50.793773 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f8ffb8-08bf-40aa-b875-768b9649750a None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:50.795493 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f8ffb8-08bf-40aa-b875-768b9649750a None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:50.808431 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-c3f8ffb8-08bf-40aa-b875-768b9649750a None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 24 05:57:50.808431 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:57:50.808431 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:57:50.808431 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:57:50.808431 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:57:50.808431 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:57:50.808431 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:57:50.808431 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:57:50.808431 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:57:50.808431 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:57:50.808431 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:57:50.808431 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:57:50.808431 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 05:57:50.808431 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 05:57:50.808431 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 05:57:50.808431 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 05:57:50.809523 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:57:50.809523 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:57:50.809523 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 05:57:50.809523 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 05:57:50.809523 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 05:57:50.809523 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 05:57:50.809523 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:57:50.809523 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:57:50.809523 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:57:50.809523 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:57:50.809523 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:57:50.809523 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 24 05:57:50.809523 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:57:50.809523 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:57:50.809523 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:57:50.809523 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:57:50.809523 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:57:50.809523 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:57:50.810504 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:57:50.810504 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:57:50.810504 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 05:57:50.810504 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 05:57:50.810504 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 05:57:50.810504 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 05:57:50.810504 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 05:57:50.810504 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 05:57:50.810504 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 24 05:57:50.810504 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 24 05:57:50.811070 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-901a7e09-41f1-4803-88b5-98b76097ce10 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:57:50.811148 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 318/641] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Aug 24 05:57:50 2022] GET /identity/v3/users/glance => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:57:50.811148 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f8ffb8-08bf-40aa-b875-768b9649750a None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:50.812986 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-901a7e09-41f1-4803-88b5-98b76097ce10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:57:50.815686 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:50.890366 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 324/642] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Aug 24 05:57:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 05:57:50.891439 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901a7e09-41f1-4803-88b5-98b76097ce10 None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:50.912332 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:50.936480 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f8ffb8-08bf-40aa-b875-768b9649750a None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:50.971780 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994663e2-b592-436c-ab7c-99dd60186bc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:50.972526 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994663e2-b592-436c-ab7c-99dd60186bc6 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:50.972816 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994663e2-b592-436c-ab7c-99dd60186bc6 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:50.973085 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994663e2-b592-436c-ab7c-99dd60186bc6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:50.986219 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994663e2-b592-436c-ab7c-99dd60186bc6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:50.987663 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994663e2-b592-436c-ab7c-99dd60186bc6 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:51.006072 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-994663e2-b592-436c-ab7c-99dd60186bc6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:57:51.006072 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:57:51.006072 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:57:51.006072 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:57:51.006072 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:57:51.006072 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:57:51.006072 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:57:51.006072 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:57:51.006072 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:57:51.006072 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:57:51.006072 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:57:51.006072 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:57:51.006072 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 05:57:51.006072 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 05:57:51.006072 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 05:57:51.006072 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 05:57:51.007084 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:57:51.007084 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:57:51.007084 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:57:51.007084 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:57:51.007084 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:57:51.007084 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:57:51.007084 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:57:51.007084 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 24 05:57:51.007084 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:57:51.007084 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:57:51.007084 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:57:51.007084 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:57:51.007084 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:57:51.007084 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:57:51.007084 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:57:51.007084 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:57:51.007084 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 05:57:51.007985 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 05:57:51.007985 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:57:51.007985 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 24 05:57:51.007985 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 319/643] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Aug 24 05:57:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:57:51.008378 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994663e2-b592-436c-ab7c-99dd60186bc6 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:51.022385 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901a7e09-41f1-4803-88b5-98b76097ce10 None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:51.074778 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:51.079717 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b79695c-b316-43dc-8a01-46b9ae764f9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:51.080700 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b79695c-b316-43dc-8a01-46b9ae764f9f None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:51.081307 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b79695c-b316-43dc-8a01-46b9ae764f9f None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:51.082031 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b79695c-b316-43dc-8a01-46b9ae764f9f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:51.095424 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b79695c-b316-43dc-8a01-46b9ae764f9f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:51.097569 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b79695c-b316-43dc-8a01-46b9ae764f9f None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:51.114262 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application [None req-3b79695c-b316-43dc-8a01-46b9ae764f9f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:57:51.114262 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:57:51.114262 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:57:51.114262 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:57:51.114262 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:57:51.114262 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:57:51.114262 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:57:51.114262 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:57:51.114262 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:57:51.114262 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:57:51.114262 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:57:51.114262 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:57:51.114262 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 05:57:51.114262 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 05:57:51.114262 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 05:57:51.114262 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 05:57:51.115457 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:57:51.115457 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:57:51.115457 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:57:51.115457 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:57:51.115457 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:57:51.115457 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:57:51.115457 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:57:51.115457 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application with Lock( Aug 24 05:57:51.115457 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:57:51.115457 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:57:51.115457 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:57:51.115457 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:57:51.115457 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:57:51.115457 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:57:51.115457 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:57:51.115457 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:57:51.115457 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 05:57:51.116711 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 05:57:51.116711 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:57:51.116711 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Aug 24 05:57:51.120033 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 325/644] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Aug 24 05:57:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:57:51.120892 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b79695c-b316-43dc-8a01-46b9ae764f9f None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:51.131228 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994663e2-b592-436c-ab7c-99dd60186bc6 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:51.169088 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b8e221-d7c7-4168-9d78-cf8f5e9cb06c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:51.170020 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b8e221-d7c7-4168-9d78-cf8f5e9cb06c None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:51.170826 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b8e221-d7c7-4168-9d78-cf8f5e9cb06c None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:51.171240 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b8e221-d7c7-4168-9d78-cf8f5e9cb06c None admin] PATH_INFO: `/v3/users` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:51.172717 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b8e221-d7c7-4168-9d78-cf8f5e9cb06c None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:57:51.173166 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b8e221-d7c7-4168-9d78-cf8f5e9cb06c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:51.174837 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b8e221-d7c7-4168-9d78-cf8f5e9cb06c None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:51.183569 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:51.192898 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 320/645] 10.222.0.62 () {56 vars in 1137 bytes} [Wed Aug 24 05:57:51 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:57:51.193920 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b8e221-d7c7-4168-9d78-cf8f5e9cb06c None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:51.225471 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b79695c-b316-43dc-8a01-46b9ae764f9f None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:51.260750 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07631c5-ba6f-40b1-b1c0-6232946919ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:51.261827 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07631c5-ba6f-40b1-b1c0-6232946919ce None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:51.262116 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07631c5-ba6f-40b1-b1c0-6232946919ce None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:51.262385 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07631c5-ba6f-40b1-b1c0-6232946919ce None admin] PATH_INFO: `/v3/domains` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:51.263019 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07631c5-ba6f-40b1-b1c0-6232946919ce None admin] RBAC: Adding query filter params (name=Default) {{(pid=81851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:57:51.263318 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07631c5-ba6f-40b1-b1c0-6232946919ce None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:51.264703 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07631c5-ba6f-40b1-b1c0-6232946919ce None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:51.276572 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 326/646] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Aug 24 05:57:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:57:51.277379 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07631c5-ba6f-40b1-b1c0-6232946919ce None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:51.300399 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b8e221-d7c7-4168-9d78-cf8f5e9cb06c None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:51.315616 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:51.337935 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7760bf6-575a-4eaa-b709-8679679d45c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:51.338466 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7760bf6-575a-4eaa-b709-8679679d45c1 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:51.338742 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7760bf6-575a-4eaa-b709-8679679d45c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:51.339009 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7760bf6-575a-4eaa-b709-8679679d45c1 None admin] PATH_INFO: `/v3/domains` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:51.339637 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7760bf6-575a-4eaa-b709-8679679d45c1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:57:51.339931 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7760bf6-575a-4eaa-b709-8679679d45c1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:51.341381 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7760bf6-575a-4eaa-b709-8679679d45c1 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:51.352907 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 321/647] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Aug 24 05:57:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:57:51.358486 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7760bf6-575a-4eaa-b709-8679679d45c1 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:51.388242 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07631c5-ba6f-40b1-b1c0-6232946919ce None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:51.413816 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:51.424530 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d9532e-d126-4541-9360-b55635d74bb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:51.430628 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d9532e-d126-4541-9360-b55635d74bb8 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:51.430628 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d9532e-d126-4541-9360-b55635d74bb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:51.430628 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d9532e-d126-4541-9360-b55635d74bb8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:51.436149 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d9532e-d126-4541-9360-b55635d74bb8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:57:51.440938 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application [None req-58d9532e-d126-4541-9360-b55635d74bb8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 05:57:51.440938 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:57:51.440938 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:57:51.440938 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:57:51.440938 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:57:51.440938 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:57:51.440938 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:57:51.440938 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:57:51.440938 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:57:51.440938 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:57:51.440938 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:57:51.440938 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:57:51.440938 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:57:51.440938 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:57:51.440938 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:57:51.440938 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:57:51.441908 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:57:51.441908 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:57:51.441908 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:57:51.441908 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:57:51.441908 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:57:51.441908 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:57:51.441908 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:57:51.441908 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:57:51.441908 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:57:51.441908 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:57:51.441908 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:57:51.441908 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application with Lock( Aug 24 05:57:51.441908 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:57:51.441908 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:57:51.441908 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:57:51.441908 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:57:51.445602 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:57:51.445602 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:57:51.445602 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:57:51.445602 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:57:51.445602 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:57:51.445602 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:57:51.445602 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:57:51.445602 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:57:51.445602 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:57:51.445602 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:57:51.445602 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 05:57:51.445602 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Aug 24 05:57:51.445602 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 327/648] 10.222.0.62 () {56 vars in 1102 bytes} [Wed Aug 24 05:57:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:57:51.445602 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d9532e-d126-4541-9360-b55635d74bb8 None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:51.475229 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7760bf6-575a-4eaa-b709-8679679d45c1 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:51.509751 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46426d8f-3bf8-46dc-aff9-b8c88620a2e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:51.510348 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46426d8f-3bf8-46dc-aff9-b8c88620a2e3 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:51.510624 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46426d8f-3bf8-46dc-aff9-b8c88620a2e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:51.510891 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46426d8f-3bf8-46dc-aff9-b8c88620a2e3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:51.552922 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:51.558489 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d9532e-d126-4541-9360-b55635d74bb8 None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:51.580962 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46426d8f-3bf8-46dc-aff9-b8c88620a2e3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:51.582637 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46426d8f-3bf8-46dc-aff9-b8c88620a2e3 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:51.594285 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3164da08-46b7-4513-8514-7b8e9d63e180 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:51.594846 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3164da08-46b7-4513-8514-7b8e9d63e180 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:51.595125 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3164da08-46b7-4513-8514-7b8e9d63e180 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:51.595392 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3164da08-46b7-4513-8514-7b8e9d63e180 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:51.602904 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3164da08-46b7-4513-8514-7b8e9d63e180 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:51.604411 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3164da08-46b7-4513-8514-7b8e9d63e180 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:51.616434 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application [None req-3164da08-46b7-4513-8514-7b8e9d63e180 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 05:57:51.616434 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:57:51.616434 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:57:51.616434 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:57:51.616434 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:57:51.616434 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:57:51.616434 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:57:51.616434 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:57:51.616434 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:57:51.616434 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:57:51.616434 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:57:51.616434 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:57:51.616434 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 05:57:51.616434 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 05:57:51.616434 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 05:57:51.616434 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 05:57:51.617410 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:57:51.617410 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:57:51.617410 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 05:57:51.617410 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 05:57:51.617410 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 05:57:51.617410 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 05:57:51.617410 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:57:51.617410 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:57:51.617410 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:57:51.617410 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:57:51.617410 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:57:51.617410 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application with Lock( Aug 24 05:57:51.617410 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:57:51.617410 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:57:51.617410 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:57:51.617410 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:57:51.617410 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:57:51.617410 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:57:51.618351 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:57:51.618351 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:57:51.618351 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 05:57:51.618351 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 05:57:51.618351 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 05:57:51.618351 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 05:57:51.618351 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 05:57:51.618351 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 05:57:51.618351 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 05:57:51.618351 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Aug 24 05:57:51.618351 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 328/649] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Aug 24 05:57:51 2022] GET /identity/v3/users/cinder => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:57:51.618351 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3164da08-46b7-4513-8514-7b8e9d63e180 None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:51.667792 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:51.691676 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 322/650] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Aug 24 05:57:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 05:57:51.692797 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46426d8f-3bf8-46dc-aff9-b8c88620a2e3 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:51.729343 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3164da08-46b7-4513-8514-7b8e9d63e180 None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:51.761805 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66026a9d-15de-4f5c-86f7-2ef5562e4e1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:51.762459 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66026a9d-15de-4f5c-86f7-2ef5562e4e1c None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:51.762739 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66026a9d-15de-4f5c-86f7-2ef5562e4e1c None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:51.763011 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66026a9d-15de-4f5c-86f7-2ef5562e4e1c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:51.764791 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:51.771235 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66026a9d-15de-4f5c-86f7-2ef5562e4e1c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:51.772798 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66026a9d-15de-4f5c-86f7-2ef5562e4e1c None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:51.784101 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application [None req-66026a9d-15de-4f5c-86f7-2ef5562e4e1c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 05:57:51.784101 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:57:51.784101 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:57:51.784101 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:57:51.784101 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:57:51.784101 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:57:51.784101 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:57:51.784101 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:57:51.784101 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:57:51.784101 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:57:51.784101 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:57:51.784101 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:57:51.784101 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 05:57:51.784101 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 05:57:51.784101 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 05:57:51.785078 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 05:57:51.785078 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:57:51.785078 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:57:51.785078 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:57:51.785078 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:57:51.785078 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:57:51.785078 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:57:51.785078 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:57:51.785078 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application with Lock( Aug 24 05:57:51.785078 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:57:51.785078 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:57:51.785078 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:57:51.785078 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:57:51.785078 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:57:51.785078 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:57:51.785078 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:57:51.785078 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:57:51.785952 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 05:57:51.785952 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 05:57:51.785952 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 05:57:51.785952 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 05:57:51.785952 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 05:57:51.785952 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 05:57:51.785952 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 05:57:51.785952 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Aug 24 05:57:51.785952 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 329/651] 10.222.0.62 () {56 vars in 1111 bytes} [Wed Aug 24 05:57:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:57:51.786434 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66026a9d-15de-4f5c-86f7-2ef5562e4e1c None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:51.799054 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46426d8f-3bf8-46dc-aff9-b8c88620a2e3 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:51.833955 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc73cf0-b8f7-4e15-83bc-e08b17095096 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:51.834555 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc73cf0-b8f7-4e15-83bc-e08b17095096 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:51.834835 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc73cf0-b8f7-4e15-83bc-e08b17095096 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:51.835106 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc73cf0-b8f7-4e15-83bc-e08b17095096 None admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:51.835772 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc73cf0-b8f7-4e15-83bc-e08b17095096 None admin] RBAC: Adding query filter params (name=service) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:57:51.836071 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc73cf0-b8f7-4e15-83bc-e08b17095096 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:51.837464 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc73cf0-b8f7-4e15-83bc-e08b17095096 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:51.845549 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 323/652] 10.222.0.62 () {56 vars in 1103 bytes} [Wed Aug 24 05:57:51 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:57:51.846720 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c002584-3965-45e8-8892-5e454be41ad6 None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:51.847005 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c002584-3965-45e8-8892-5e454be41ad6 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:51.847270 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c002584-3965-45e8-8892-5e454be41ad6 None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:51.848646 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 324/653] 10.222.0.62 () {54 vars in 927 bytes} [Wed Aug 24 05:57:51 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:57:51.849157 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c002584-3965-45e8-8892-5e454be41ad6 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:51.892947 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66026a9d-15de-4f5c-86f7-2ef5562e4e1c None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:51.895301 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:51.927812 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620f74bc-bd24-445e-a8aa-3594a7a111fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:51.928412 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620f74bc-bd24-445e-a8aa-3594a7a111fe None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:51.928723 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620f74bc-bd24-445e-a8aa-3594a7a111fe None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:51.929055 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620f74bc-bd24-445e-a8aa-3594a7a111fe None admin] PATH_INFO: `/v3/users` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:51.930028 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620f74bc-bd24-445e-a8aa-3594a7a111fe None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:57:51.930563 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620f74bc-bd24-445e-a8aa-3594a7a111fe None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:51.932317 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620f74bc-bd24-445e-a8aa-3594a7a111fe None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:51.948132 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c002584-3965-45e8-8892-5e454be41ad6 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:51.949733 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 330/654] 10.222.0.62 () {56 vars in 1137 bytes} [Wed Aug 24 05:57:51 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:57:51.950611 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620f74bc-bd24-445e-a8aa-3594a7a111fe None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:51.985484 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14df0105-6dd9-49b0-85a0-8f1a5d6ee300 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:51.986101 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14df0105-6dd9-49b0-85a0-8f1a5d6ee300 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:51.986378 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14df0105-6dd9-49b0-85a0-8f1a5d6ee300 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:51.986646 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14df0105-6dd9-49b0-85a0-8f1a5d6ee300 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:51.988505 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:51.997280 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14df0105-6dd9-49b0-85a0-8f1a5d6ee300 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:51.998731 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14df0105-6dd9-49b0-85a0-8f1a5d6ee300 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:52.012809 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-14df0105-6dd9-49b0-85a0-8f1a5d6ee300 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:57:52.012809 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:57:52.012809 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:57:52.012809 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:57:52.012809 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:57:52.012809 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:57:52.012809 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:57:52.012809 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:57:52.012809 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:57:52.012809 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:57:52.012809 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:57:52.012809 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:57:52.012809 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 05:57:52.012809 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 05:57:52.012809 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 05:57:52.012809 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 05:57:52.013928 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:57:52.013928 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:57:52.013928 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:57:52.013928 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:57:52.013928 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:57:52.013928 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:57:52.013928 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:57:52.013928 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 24 05:57:52.013928 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:57:52.013928 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:57:52.013928 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:57:52.013928 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:57:52.013928 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:57:52.013928 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:57:52.013928 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:57:52.013928 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:57:52.013928 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 05:57:52.014827 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 05:57:52.014827 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:57:52.014827 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 24 05:57:52.014827 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 325/655] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Aug 24 05:57:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:57:52.014827 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14df0105-6dd9-49b0-85a0-8f1a5d6ee300 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:52.050950 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620f74bc-bd24-445e-a8aa-3594a7a111fe None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:52.085766 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9e5b30-3d7a-4802-9a37-dd5daf03e14b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:52.086375 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9e5b30-3d7a-4802-9a37-dd5daf03e14b None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:52.086654 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9e5b30-3d7a-4802-9a37-dd5daf03e14b None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:52.086936 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9e5b30-3d7a-4802-9a37-dd5daf03e14b None admin] PATH_INFO: `/v3/projects` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:52.087619 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9e5b30-3d7a-4802-9a37-dd5daf03e14b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:57:52.087925 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9e5b30-3d7a-4802-9a37-dd5daf03e14b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:52.089393 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9e5b30-3d7a-4802-9a37-dd5daf03e14b None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:52.100719 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 331/656] 10.222.0.62 () {56 vars in 1148 bytes} [Wed Aug 24 05:57:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:57:52.101816 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e08e57d-9654-426a-82b8-9cbbbf37c48b None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:52.102100 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e08e57d-9654-426a-82b8-9cbbbf37c48b None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:52.102371 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e08e57d-9654-426a-82b8-9cbbbf37c48b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:52.119427 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-8e08e57d-9654-426a-82b8-9cbbbf37c48b None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:57:52.120621 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-8e08e57d-9654-426a-82b8-9cbbbf37c48b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:57:52.121724 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14df0105-6dd9-49b0-85a0-8f1a5d6ee300 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:52.137282 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:52.155698 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7b0895-702b-48cc-80cc-7574b72220ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:52.156317 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7b0895-702b-48cc-80cc-7574b72220ac None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:52.156628 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7b0895-702b-48cc-80cc-7574b72220ac None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:52.156906 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7b0895-702b-48cc-80cc-7574b72220ac None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:52.167530 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7b0895-702b-48cc-80cc-7574b72220ac None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:52.169023 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7b0895-702b-48cc-80cc-7574b72220ac None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:52.183805 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-ec7b0895-702b-48cc-80cc-7574b72220ac None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:57:52.183805 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:57:52.183805 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:57:52.183805 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:57:52.183805 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:57:52.183805 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:57:52.183805 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:57:52.183805 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:57:52.183805 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:57:52.183805 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:57:52.183805 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:57:52.183805 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:57:52.183805 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 05:57:52.183805 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 05:57:52.183805 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 05:57:52.183805 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 05:57:52.184939 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:57:52.184939 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:57:52.184939 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:57:52.184939 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:57:52.184939 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:57:52.184939 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:57:52.184939 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:57:52.184939 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 24 05:57:52.184939 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:57:52.184939 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:57:52.184939 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:57:52.184939 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:57:52.184939 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:57:52.184939 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:57:52.184939 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:57:52.184939 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:57:52.184939 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 05:57:52.186381 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 05:57:52.186381 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:57:52.186381 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 24 05:57:52.186381 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 326/657] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Aug 24 05:57:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:57:52.186381 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7b0895-702b-48cc-80cc-7574b72220ac None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:52.196617 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 332/658] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Aug 24 05:57:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 05:57:52.196617 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e08e57d-9654-426a-82b8-9cbbbf37c48b None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:52.228977 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:52.293598 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7b0895-702b-48cc-80cc-7574b72220ac None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:52.297971 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e08e57d-9654-426a-82b8-9cbbbf37c48b None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:52.331000 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca4861a-5319-4e5f-aedc-fcf3a00ce2c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:52.331609 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca4861a-5319-4e5f-aedc-fcf3a00ce2c0 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:52.331893 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca4861a-5319-4e5f-aedc-fcf3a00ce2c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:52.332171 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca4861a-5319-4e5f-aedc-fcf3a00ce2c0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:52.333515 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f551f31-682a-4fcd-aa4b-ada3ce553f49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:52.334310 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f551f31-682a-4fcd-aa4b-ada3ce553f49 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:52.334749 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f551f31-682a-4fcd-aa4b-ada3ce553f49 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:52.335214 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f551f31-682a-4fcd-aa4b-ada3ce553f49 None admin] PATH_INFO: `/v3/domains` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:52.335997 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f551f31-682a-4fcd-aa4b-ada3ce553f49 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:57:52.336296 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f551f31-682a-4fcd-aa4b-ada3ce553f49 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:52.337698 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f551f31-682a-4fcd-aa4b-ada3ce553f49 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:52.354814 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 333/659] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Aug 24 05:57:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:57:52.355618 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f551f31-682a-4fcd-aa4b-ada3ce553f49 None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:52.370770 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:52.403082 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca4861a-5319-4e5f-aedc-fcf3a00ce2c0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:52.404567 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca4861a-5319-4e5f-aedc-fcf3a00ce2c0 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:52.443446 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:52.463309 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f551f31-682a-4fcd-aa4b-ada3ce553f49 None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:52.497398 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9b89b2-771c-45e8-bfa5-725c9de32f0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:52.497996 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9b89b2-771c-45e8-bfa5-725c9de32f0e None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:52.498305 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9b89b2-771c-45e8-bfa5-725c9de32f0e None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:52.498586 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9b89b2-771c-45e8-bfa5-725c9de32f0e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:52.499359 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9b89b2-771c-45e8-bfa5-725c9de32f0e None admin] RBAC: Adding query filter params (role.id=5d95a5fc0f324a92905533aa8d34ac15, scope.project.id=181f4762037e454489d9552cc603179c, user.id=fb6d3743340241f98cacb9e0d063298c) {{(pid=81851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:57:52.499669 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9b89b2-771c-45e8-bfa5-725c9de32f0e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:52.501147 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9b89b2-771c-45e8-bfa5-725c9de32f0e None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:52.504960 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 327/660] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Aug 24 05:57:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 05:57:52.505790 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca4861a-5319-4e5f-aedc-fcf3a00ce2c0 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:52.507232 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 334/661] 10.222.0.62 () {56 vars in 1374 bytes} [Wed Aug 24 05:57:52 2022] GET /identity/v3/role_assignments?user.id=fb6d3743340241f98cacb9e0d063298c&scope.project.id=181f4762037e454489d9552cc603179c&role.id=5d95a5fc0f324a92905533aa8d34ac15 => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:57:52.508014 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9b89b2-771c-45e8-bfa5-725c9de32f0e None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:52.602566 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:52.627585 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9b89b2-771c-45e8-bfa5-725c9de32f0e None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:52.634635 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca4861a-5319-4e5f-aedc-fcf3a00ce2c0 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:52.664691 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74138fba-3c98-4024-97ca-6430c96324d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:52.665565 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74138fba-3c98-4024-97ca-6430c96324d7 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:52.666158 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74138fba-3c98-4024-97ca-6430c96324d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:52.666436 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74138fba-3c98-4024-97ca-6430c96324d7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:52.674346 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74138fba-3c98-4024-97ca-6430c96324d7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:52.676080 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1400c0df-ba7a-4c8f-989f-32e732bc7077 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:52.676247 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74138fba-3c98-4024-97ca-6430c96324d7 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:52.676712 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1400c0df-ba7a-4c8f-989f-32e732bc7077 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:52.676989 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1400c0df-ba7a-4c8f-989f-32e732bc7077 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:52.677257 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1400c0df-ba7a-4c8f-989f-32e732bc7077 None admin] PATH_INFO: `/v3/domains` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:52.677896 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1400c0df-ba7a-4c8f-989f-32e732bc7077 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:57:52.678361 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1400c0df-ba7a-4c8f-989f-32e732bc7077 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:52.680049 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1400c0df-ba7a-4c8f-989f-32e732bc7077 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:52.689326 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:52.689627 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application [None req-74138fba-3c98-4024-97ca-6430c96324d7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 05:57:52.689627 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:57:52.689627 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:57:52.689627 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:57:52.689627 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:57:52.689627 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:57:52.689627 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:57:52.689627 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:57:52.689627 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:57:52.689627 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:57:52.689627 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:57:52.689627 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:57:52.689627 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 05:57:52.689627 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 05:57:52.689627 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 05:57:52.690610 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 05:57:52.690610 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:57:52.690610 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:57:52.690610 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:57:52.690610 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:57:52.690610 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:57:52.690610 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:57:52.690610 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:57:52.690610 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application with Lock( Aug 24 05:57:52.690610 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:57:52.690610 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:57:52.690610 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:57:52.690610 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:57:52.690610 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:57:52.690610 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:57:52.690610 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:57:52.690610 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:57:52.691574 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 05:57:52.691574 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 05:57:52.691574 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 05:57:52.691574 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 05:57:52.691574 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 05:57:52.691574 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 05:57:52.691574 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 05:57:52.691574 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Aug 24 05:57:52.691574 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 335/662] 10.222.0.62 () {56 vars in 1111 bytes} [Wed Aug 24 05:57:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:57:52.692093 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74138fba-3c98-4024-97ca-6430c96324d7 None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:52.697948 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 328/663] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Aug 24 05:57:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:57:52.699055 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1400c0df-ba7a-4c8f-989f-32e732bc7077 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:52.800720 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74138fba-3c98-4024-97ca-6430c96324d7 None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:52.812702 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1400c0df-ba7a-4c8f-989f-32e732bc7077 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:52.834126 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ad62e1-57c2-4e73-934a-9380e10cce0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:52.834758 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ad62e1-57c2-4e73-934a-9380e10cce0a None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:52.835034 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ad62e1-57c2-4e73-934a-9380e10cce0a None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:52.835303 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ad62e1-57c2-4e73-934a-9380e10cce0a None admin] PATH_INFO: `/v3/roles/service` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:52.845821 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ad62e1-57c2-4e73-934a-9380e10cce0a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:57:52.847910 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14f469e-572a-416c-b482-e1aeb3283456 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:52.848704 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14f469e-572a-416c-b482-e1aeb3283456 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:52.848986 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14f469e-572a-416c-b482-e1aeb3283456 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:52.849259 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14f469e-572a-416c-b482-e1aeb3283456 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:52.850896 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application [None req-76ad62e1-57c2-4e73-934a-9380e10cce0a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 05:57:52.850896 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:57:52.850896 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:57:52.850896 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:57:52.850896 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:57:52.850896 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:57:52.850896 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:57:52.850896 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:57:52.850896 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:57:52.850896 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:57:52.850896 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:57:52.850896 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:57:52.850896 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:57:52.850896 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:57:52.850896 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:57:52.850896 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:57:52.856758 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:57:52.856758 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:57:52.856758 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:57:52.856758 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:57:52.856758 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:57:52.856758 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:57:52.856758 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:57:52.856758 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:57:52.856758 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:57:52.856758 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:57:52.856758 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:57:52.856758 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application with Lock( Aug 24 05:57:52.856758 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:57:52.856758 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:57:52.856758 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:57:52.856758 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:57:52.857890 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:57:52.857890 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:57:52.857890 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:57:52.857890 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:57:52.857890 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:57:52.857890 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:57:52.857890 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:57:52.857890 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:57:52.857890 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:57:52.857890 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:57:52.857890 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 05:57:52.857890 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Aug 24 05:57:52.857890 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 336/664] 10.222.0.62 () {56 vars in 1102 bytes} [Wed Aug 24 05:57:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:57:52.857890 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ad62e1-57c2-4e73-934a-9380e10cce0a None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:52.861605 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14f469e-572a-416c-b482-e1aeb3283456 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:52.863160 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14f469e-572a-416c-b482-e1aeb3283456 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:52.877788 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-a14f469e-572a-416c-b482-e1aeb3283456 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:57:52.877788 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:57:52.877788 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:57:52.877788 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:57:52.877788 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:57:52.877788 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:57:52.877788 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:57:52.877788 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:57:52.877788 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:57:52.877788 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:57:52.877788 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:57:52.877788 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:57:52.877788 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 05:57:52.877788 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 05:57:52.877788 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 05:57:52.877788 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 05:57:52.878741 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:57:52.878741 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:57:52.878741 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:57:52.878741 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:57:52.878741 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:57:52.878741 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:57:52.878741 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:57:52.878741 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 24 05:57:52.878741 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:57:52.878741 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:57:52.878741 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:57:52.878741 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:57:52.878741 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:57:52.878741 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:57:52.878741 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:57:52.878741 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:57:52.878741 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 05:57:52.879637 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 05:57:52.879637 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:57:52.879637 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 24 05:57:52.879637 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 329/665] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Aug 24 05:57:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:57:52.880299 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14f469e-572a-416c-b482-e1aeb3283456 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:52.925383 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:52.963165 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ad62e1-57c2-4e73-934a-9380e10cce0a None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:52.982160 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:52.988764 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14f469e-572a-416c-b482-e1aeb3283456 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:52.997162 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe231571-99e6-41bb-a76d-d983e0f867dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:52.997773 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe231571-99e6-41bb-a76d-d983e0f867dc None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:52.998049 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe231571-99e6-41bb-a76d-d983e0f867dc None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:52.998316 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe231571-99e6-41bb-a76d-d983e0f867dc None admin] PATH_INFO: `/v3/projects` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:52.999007 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe231571-99e6-41bb-a76d-d983e0f867dc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:57:52.999325 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe231571-99e6-41bb-a76d-d983e0f867dc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:53.000799 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe231571-99e6-41bb-a76d-d983e0f867dc None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:53.014284 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 337/666] 10.222.0.62 () {56 vars in 1148 bytes} [Wed Aug 24 05:57:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:57:53.015730 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe231571-99e6-41bb-a76d-d983e0f867dc None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:53.027000 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2fff2c-0cfe-4997-a485-e05cf8942f17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:53.028275 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2fff2c-0cfe-4997-a485-e05cf8942f17 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:53.028828 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2fff2c-0cfe-4997-a485-e05cf8942f17 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:53.029325 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2fff2c-0cfe-4997-a485-e05cf8942f17 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:53.039263 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2fff2c-0cfe-4997-a485-e05cf8942f17 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:53.040728 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2fff2c-0cfe-4997-a485-e05cf8942f17 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:53.052146 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-ed2fff2c-0cfe-4997-a485-e05cf8942f17 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 24 05:57:53.052146 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:57:53.052146 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:57:53.052146 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:57:53.052146 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:57:53.052146 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:57:53.052146 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:57:53.052146 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:57:53.052146 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:57:53.052146 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:57:53.052146 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:57:53.052146 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:57:53.052146 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 05:57:53.052146 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 05:57:53.052146 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 05:57:53.052146 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 05:57:53.053180 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:57:53.053180 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:57:53.053180 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 05:57:53.053180 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 05:57:53.053180 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 05:57:53.053180 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 05:57:53.053180 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:57:53.053180 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:57:53.053180 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:57:53.053180 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:57:53.053180 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:57:53.053180 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 24 05:57:53.053180 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:57:53.053180 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:57:53.053180 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:57:53.053180 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:57:53.053180 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:57:53.053180 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:57:53.054140 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:57:53.054140 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:57:53.054140 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 05:57:53.054140 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 05:57:53.054140 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 05:57:53.054140 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 05:57:53.054140 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 05:57:53.054140 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 05:57:53.054140 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 24 05:57:53.054140 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 24 05:57:53.054140 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 330/667] 10.222.0.62 () {56 vars in 1102 bytes} [Wed Aug 24 05:57:52 2022] GET /identity/v3/users/neutron => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:57:53.054140 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2fff2c-0cfe-4997-a485-e05cf8942f17 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:53.084815 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:53.131549 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe231571-99e6-41bb-a76d-d983e0f867dc None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:53.160206 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2fff2c-0cfe-4997-a485-e05cf8942f17 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:53.171019 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1caf1b-0b4b-4788-a87a-8efff266eaf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:53.171519 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1caf1b-0b4b-4788-a87a-8efff266eaf3 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:53.171797 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1caf1b-0b4b-4788-a87a-8efff266eaf3 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:53.171916 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:53.172055 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1caf1b-0b4b-4788-a87a-8efff266eaf3 None admin] PATH_INFO: `/v3/roles` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:53.172744 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1caf1b-0b4b-4788-a87a-8efff266eaf3 None admin] RBAC: Adding query filter params (name=service) {{(pid=81851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:57:53.173044 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1caf1b-0b4b-4788-a87a-8efff266eaf3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:53.174396 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1caf1b-0b4b-4788-a87a-8efff266eaf3 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:53.187419 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 338/668] 10.222.0.62 () {56 vars in 1103 bytes} [Wed Aug 24 05:57:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:57:53.187419 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1caf1b-0b4b-4788-a87a-8efff266eaf3 None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:53.202157 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5244435-b4b2-406e-9644-015b48dec39c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:53.203539 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5244435-b4b2-406e-9644-015b48dec39c None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:53.204403 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5244435-b4b2-406e-9644-015b48dec39c None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:53.204555 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5244435-b4b2-406e-9644-015b48dec39c None admin] PATH_INFO: `/v3/domains` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:53.205133 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5244435-b4b2-406e-9644-015b48dec39c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:57:53.205643 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5244435-b4b2-406e-9644-015b48dec39c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:53.207685 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5244435-b4b2-406e-9644-015b48dec39c None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:53.220070 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 331/669] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Aug 24 05:57:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:57:53.220850 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5244435-b4b2-406e-9644-015b48dec39c None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:53.293776 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1caf1b-0b4b-4788-a87a-8efff266eaf3 None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:53.310976 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:53.324776 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5244435-b4b2-406e-9644-015b48dec39c None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:53.330459 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa00d9b-6736-41ab-baa2-618a7dd26010 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:53.331058 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa00d9b-6736-41ab-baa2-618a7dd26010 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:53.331333 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa00d9b-6736-41ab-baa2-618a7dd26010 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:53.331601 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa00d9b-6736-41ab-baa2-618a7dd26010 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:53.332310 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa00d9b-6736-41ab-baa2-618a7dd26010 None admin] RBAC: Adding query filter params (role.id=5d95a5fc0f324a92905533aa8d34ac15, scope.project.id=181f4762037e454489d9552cc603179c, user.id=c7d52162cbdc4218b577ecc1e69a88c9) {{(pid=81851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:57:53.333044 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa00d9b-6736-41ab-baa2-618a7dd26010 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:53.334809 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa00d9b-6736-41ab-baa2-618a7dd26010 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:53.342664 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 339/670] 10.222.0.62 () {56 vars in 1374 bytes} [Wed Aug 24 05:57:53 2022] GET /identity/v3/role_assignments?user.id=c7d52162cbdc4218b577ecc1e69a88c9&scope.project.id=181f4762037e454489d9552cc603179c&role.id=5d95a5fc0f324a92905533aa8d34ac15 => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:57:53.343461 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa00d9b-6736-41ab-baa2-618a7dd26010 None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:53.360222 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd21f26-595c-4282-bf55-0c402b4e1b3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:53.361308 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd21f26-595c-4282-bf55-0c402b4e1b3d None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:53.361657 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd21f26-595c-4282-bf55-0c402b4e1b3d None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:53.362053 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd21f26-595c-4282-bf55-0c402b4e1b3d None admin] PATH_INFO: `/v3/users` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:53.362978 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd21f26-595c-4282-bf55-0c402b4e1b3d None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:57:53.363359 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd21f26-595c-4282-bf55-0c402b4e1b3d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:53.364940 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd21f26-595c-4282-bf55-0c402b4e1b3d None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:53.389136 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 332/671] 10.222.0.62 () {56 vars in 1139 bytes} [Wed Aug 24 05:57:53 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:57:53.389136 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd21f26-595c-4282-bf55-0c402b4e1b3d None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:53.420404 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:53.481712 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa00d9b-6736-41ab-baa2-618a7dd26010 None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:53.505565 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd21f26-595c-4282-bf55-0c402b4e1b3d None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:53.526290 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff641600-5c1d-45af-b4cb-df1614a22737 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:53.527430 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff641600-5c1d-45af-b4cb-df1614a22737 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:53.528273 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff641600-5c1d-45af-b4cb-df1614a22737 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:53.528738 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff641600-5c1d-45af-b4cb-df1614a22737 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:53.542936 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff641600-5c1d-45af-b4cb-df1614a22737 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:53.544403 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff641600-5c1d-45af-b4cb-df1614a22737 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:53.556248 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00585cda-3deb-4c01-ae41-0bc11c4c691a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:53.557757 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00585cda-3deb-4c01-ae41-0bc11c4c691a None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:53.557757 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00585cda-3deb-4c01-ae41-0bc11c4c691a None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:53.557757 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00585cda-3deb-4c01-ae41-0bc11c4c691a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:53.566826 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application [None req-ff641600-5c1d-45af-b4cb-df1614a22737 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:57:53.566826 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:57:53.566826 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:57:53.566826 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:57:53.566826 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:57:53.566826 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:57:53.566826 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:57:53.566826 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:57:53.566826 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:57:53.566826 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:57:53.566826 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:57:53.566826 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:57:53.566826 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 05:57:53.566826 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 05:57:53.566826 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 05:57:53.566826 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 05:57:53.567823 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:57:53.567823 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:57:53.567823 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:57:53.567823 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:57:53.567823 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:57:53.567823 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:57:53.567823 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:57:53.567823 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application with Lock( Aug 24 05:57:53.567823 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:57:53.567823 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:57:53.567823 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:57:53.567823 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:57:53.567823 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:57:53.567823 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:57:53.567823 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:57:53.567823 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:57:53.567823 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 05:57:53.568835 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 05:57:53.568835 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:57:53.568835 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Aug 24 05:57:53.568835 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 340/672] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Aug 24 05:57:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:57:53.568835 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff641600-5c1d-45af-b4cb-df1614a22737 None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:53.572650 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00585cda-3deb-4c01-ae41-0bc11c4c691a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:53.574357 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00585cda-3deb-4c01-ae41-0bc11c4c691a None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:53.586798 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-00585cda-3deb-4c01-ae41-0bc11c4c691a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 05:57:53.586798 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:57:53.586798 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:57:53.586798 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:57:53.586798 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:57:53.586798 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:57:53.586798 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:57:53.586798 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:57:53.586798 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:57:53.586798 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:57:53.586798 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:57:53.586798 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:57:53.586798 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 05:57:53.586798 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 05:57:53.586798 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 05:57:53.587809 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 05:57:53.587809 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:57:53.587809 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:57:53.587809 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:57:53.587809 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:57:53.587809 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:57:53.587809 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:57:53.587809 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:57:53.587809 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 24 05:57:53.587809 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:57:53.587809 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:57:53.587809 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:57:53.587809 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:57:53.587809 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:57:53.587809 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:57:53.587809 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:57:53.587809 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:57:53.588825 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 05:57:53.588825 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 05:57:53.588825 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 05:57:53.588825 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 05:57:53.588825 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 05:57:53.588825 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 05:57:53.588825 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 05:57:53.588825 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 24 05:57:53.588825 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 333/673] 10.222.0.62 () {56 vars in 1111 bytes} [Wed Aug 24 05:57:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:57:53.589930 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00585cda-3deb-4c01-ae41-0bc11c4c691a None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:53.614579 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:53.696346 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:53.714539 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00585cda-3deb-4c01-ae41-0bc11c4c691a None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:53.720405 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff641600-5c1d-45af-b4cb-df1614a22737 None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:53.757466 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e1c605-efce-4b2e-ad12-f2c8579dcda8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:53.758208 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e1c605-efce-4b2e-ad12-f2c8579dcda8 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:53.758620 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e1c605-efce-4b2e-ad12-f2c8579dcda8 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:53.758898 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e1c605-efce-4b2e-ad12-f2c8579dcda8 None admin] PATH_INFO: `/v3/domains` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:53.759651 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e1c605-efce-4b2e-ad12-f2c8579dcda8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:57:53.759953 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e1c605-efce-4b2e-ad12-f2c8579dcda8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:53.761347 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e1c605-efce-4b2e-ad12-f2c8579dcda8 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:53.763314 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1cb000-f0c0-4a4d-b08d-0d7dfff91ff6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:53.764644 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1cb000-f0c0-4a4d-b08d-0d7dfff91ff6 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:53.765111 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1cb000-f0c0-4a4d-b08d-0d7dfff91ff6 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:53.765523 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1cb000-f0c0-4a4d-b08d-0d7dfff91ff6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:53.774906 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 334/674] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Aug 24 05:57:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:57:53.780971 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e1c605-efce-4b2e-ad12-f2c8579dcda8 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:53.857905 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1cb000-f0c0-4a4d-b08d-0d7dfff91ff6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:53.860143 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1cb000-f0c0-4a4d-b08d-0d7dfff91ff6 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:53.889968 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e1c605-efce-4b2e-ad12-f2c8579dcda8 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:53.902424 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:53.935312 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2e1807-9774-47b9-838e-46308219ba41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:53.936290 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2e1807-9774-47b9-838e-46308219ba41 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:53.936782 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2e1807-9774-47b9-838e-46308219ba41 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:53.937391 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2e1807-9774-47b9-838e-46308219ba41 None admin] PATH_INFO: `/v3/projects` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:53.938647 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2e1807-9774-47b9-838e-46308219ba41 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:57:53.939234 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2e1807-9774-47b9-838e-46308219ba41 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:53.940948 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2e1807-9774-47b9-838e-46308219ba41 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:53.957625 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 335/675] 10.222.0.62 () {56 vars in 1148 bytes} [Wed Aug 24 05:57:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:57:53.958723 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2e1807-9774-47b9-838e-46308219ba41 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:53.971989 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:53.988673 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 341/676] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Aug 24 05:57:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 05:57:53.989735 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1cb000-f0c0-4a4d-b08d-0d7dfff91ff6 None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:54.087472 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2e1807-9774-47b9-838e-46308219ba41 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:54.108651 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1cb000-f0c0-4a4d-b08d-0d7dfff91ff6 None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:54.132019 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a15559-af26-4935-96e1-0401893a14ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:54.133040 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a15559-af26-4935-96e1-0401893a14ab None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:54.133786 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a15559-af26-4935-96e1-0401893a14ab None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:54.134452 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a15559-af26-4935-96e1-0401893a14ab None admin] PATH_INFO: `/v3/users/swift` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:54.145832 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a15559-af26-4935-96e1-0401893a14ab None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:54.147268 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a15559-af26-4935-96e1-0401893a14ab None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:54.151181 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e0f5db-f80d-4833-99e3-814f4dba1126 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:54.152048 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e0f5db-f80d-4833-99e3-814f4dba1126 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:54.152561 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e0f5db-f80d-4833-99e3-814f4dba1126 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:54.153042 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e0f5db-f80d-4833-99e3-814f4dba1126 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:54.154256 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e0f5db-f80d-4833-99e3-814f4dba1126 None admin] RBAC: Adding query filter params (role.id=5d95a5fc0f324a92905533aa8d34ac15, scope.project.id=181f4762037e454489d9552cc603179c, user.id=8c4def55ac0e48b29836bc5c64bc7402) {{(pid=81851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:57:54.154790 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e0f5db-f80d-4833-99e3-814f4dba1126 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:54.156437 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e0f5db-f80d-4833-99e3-814f4dba1126 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:54.158563 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:54.159820 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-34a15559-af26-4935-96e1-0401893a14ab None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 24 05:57:54.159820 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:57:54.159820 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:57:54.159820 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:57:54.159820 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:57:54.159820 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:57:54.159820 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:57:54.159820 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:57:54.159820 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:57:54.159820 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:57:54.159820 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:57:54.159820 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:57:54.159820 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 05:57:54.159820 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 05:57:54.159820 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 05:57:54.159820 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 05:57:54.160780 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:57:54.160780 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:57:54.160780 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 05:57:54.160780 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 05:57:54.160780 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 05:57:54.160780 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 05:57:54.160780 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:57:54.160780 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:57:54.160780 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:57:54.160780 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:57:54.160780 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:57:54.160780 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 24 05:57:54.160780 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:57:54.160780 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:57:54.160780 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:57:54.160780 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:57:54.160780 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:57:54.160780 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:57:54.162298 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:57:54.162298 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:57:54.162298 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 05:57:54.162298 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 05:57:54.162298 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 05:57:54.162298 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 05:57:54.162298 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 05:57:54.162298 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 05:57:54.162298 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 24 05:57:54.162298 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 24 05:57:54.162298 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 336/677] 10.222.0.62 () {56 vars in 1096 bytes} [Wed Aug 24 05:57:53 2022] GET /identity/v3/users/swift => generated 83 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:57:54.162298 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a15559-af26-4935-96e1-0401893a14ab None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:54.168963 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 342/678] 10.222.0.62 () {56 vars in 1374 bytes} [Wed Aug 24 05:57:53 2022] GET /identity/v3/role_assignments?user.id=8c4def55ac0e48b29836bc5c64bc7402&scope.project.id=181f4762037e454489d9552cc603179c&role.id=5d95a5fc0f324a92905533aa8d34ac15 => generated 608 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:57:54.169991 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e0f5db-f80d-4833-99e3-814f4dba1126 None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:54.292530 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e0f5db-f80d-4833-99e3-814f4dba1126 None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:54.306896 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:54.338752 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e4cb3e-1cb5-4f08-89ed-9c7be87ac1c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:54.340337 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e4cb3e-1cb5-4f08-89ed-9c7be87ac1c5 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:54.340845 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e4cb3e-1cb5-4f08-89ed-9c7be87ac1c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:54.341324 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e4cb3e-1cb5-4f08-89ed-9c7be87ac1c5 None admin] PATH_INFO: `/v3/users` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:54.342274 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e4cb3e-1cb5-4f08-89ed-9c7be87ac1c5 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:57:54.342793 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e4cb3e-1cb5-4f08-89ed-9c7be87ac1c5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:54.344985 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e4cb3e-1cb5-4f08-89ed-9c7be87ac1c5 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:54.367301 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 343/679] 10.222.0.62 () {56 vars in 1135 bytes} [Wed Aug 24 05:57:54 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:57:54.377604 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e4cb3e-1cb5-4f08-89ed-9c7be87ac1c5 None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:54.386955 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a15559-af26-4935-96e1-0401893a14ab None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:54.446323 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda3bc19-8ad7-4a5a-b327-21e893bbff56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:54.447374 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda3bc19-8ad7-4a5a-b327-21e893bbff56 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:54.447945 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda3bc19-8ad7-4a5a-b327-21e893bbff56 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:54.448444 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda3bc19-8ad7-4a5a-b327-21e893bbff56 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:54.459550 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:54.466261 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda3bc19-8ad7-4a5a-b327-21e893bbff56 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:54.468035 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda3bc19-8ad7-4a5a-b327-21e893bbff56 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:54.483529 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-eda3bc19-8ad7-4a5a-b327-21e893bbff56 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:57:54.483529 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:57:54.483529 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:57:54.483529 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:57:54.483529 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:57:54.483529 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:57:54.483529 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:57:54.483529 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:57:54.483529 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:57:54.483529 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:57:54.483529 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:57:54.483529 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:57:54.483529 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 05:57:54.483529 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 05:57:54.483529 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 05:57:54.483529 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 05:57:54.485072 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:57:54.485072 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:57:54.485072 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:57:54.485072 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:57:54.485072 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:57:54.485072 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:57:54.485072 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:57:54.485072 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 24 05:57:54.485072 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:57:54.485072 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:57:54.485072 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:57:54.485072 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:57:54.485072 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:57:54.485072 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:57:54.485072 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:57:54.485072 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:57:54.485072 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 05:57:54.485985 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 05:57:54.485985 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:57:54.485985 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 24 05:57:54.488607 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 337/680] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Aug 24 05:57:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 327 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:57:54.494636 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda3bc19-8ad7-4a5a-b327-21e893bbff56 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:54.528242 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e4cb3e-1cb5-4f08-89ed-9c7be87ac1c5 None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:54.576268 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f2115e-bbfb-4edc-a68a-480c7ac173c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:54.576930 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f2115e-bbfb-4edc-a68a-480c7ac173c5 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:54.577209 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f2115e-bbfb-4edc-a68a-480c7ac173c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:54.577512 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f2115e-bbfb-4edc-a68a-480c7ac173c5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:54.613897 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda3bc19-8ad7-4a5a-b327-21e893bbff56 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:54.634671 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:54.649713 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0087585-09c5-4bb5-8100-2e1985b84365 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:54.650244 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0087585-09c5-4bb5-8100-2e1985b84365 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:54.650520 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0087585-09c5-4bb5-8100-2e1985b84365 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:54.650788 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0087585-09c5-4bb5-8100-2e1985b84365 None admin] PATH_INFO: `/v3/domains` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:54.651677 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0087585-09c5-4bb5-8100-2e1985b84365 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:57:54.651796 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0087585-09c5-4bb5-8100-2e1985b84365 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:54.653210 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0087585-09c5-4bb5-8100-2e1985b84365 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:54.654053 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f2115e-bbfb-4edc-a68a-480c7ac173c5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:54.656224 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f2115e-bbfb-4edc-a68a-480c7ac173c5 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:54.672012 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 338/681] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Aug 24 05:57:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:57:54.672012 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930861dd-bf66-4d67-8fc6-3756e858acc8 None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:54.672012 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930861dd-bf66-4d67-8fc6-3756e858acc8 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:54.672012 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930861dd-bf66-4d67-8fc6-3756e858acc8 None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:54.672012 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 339/682] 10.222.0.62 () {54 vars in 927 bytes} [Wed Aug 24 05:57:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:57:54.673786 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930861dd-bf66-4d67-8fc6-3756e858acc8 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:54.731523 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:54.781270 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 344/683] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Aug 24 05:57:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 05:57:54.782853 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0840a1ca-50f2-411d-94a6-7e32e0f89f56 None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:54.783281 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0840a1ca-50f2-411d-94a6-7e32e0f89f56 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:54.783692 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0840a1ca-50f2-411d-94a6-7e32e0f89f56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:54.784038 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930861dd-bf66-4d67-8fc6-3756e858acc8 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:54.805234 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-0840a1ca-50f2-411d-94a6-7e32e0f89f56 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:57:54.806676 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-0840a1ca-50f2-411d-94a6-7e32e0f89f56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:57:54.822295 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b8add0-45c3-48e2-b586-d6e8029e98ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:54.823258 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b8add0-45c3-48e2-b586-d6e8029e98ba None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:54.823631 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b8add0-45c3-48e2-b586-d6e8029e98ba None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:54.823998 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b8add0-45c3-48e2-b586-d6e8029e98ba None admin] PATH_INFO: `/v3/projects/service` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:54.831195 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b8add0-45c3-48e2-b586-d6e8029e98ba None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:54.832794 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b8add0-45c3-48e2-b586-d6e8029e98ba None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:54.845047 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-a7b8add0-45c3-48e2-b586-d6e8029e98ba None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 05:57:54.845047 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:57:54.845047 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:57:54.845047 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:57:54.845047 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:57:54.845047 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:57:54.845047 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:57:54.845047 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:57:54.845047 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:57:54.845047 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:57:54.845047 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:57:54.845047 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:57:54.845047 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 05:57:54.845047 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 05:57:54.845047 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 05:57:54.846140 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 05:57:54.846140 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:57:54.846140 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:57:54.846140 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:57:54.846140 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:57:54.846140 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:57:54.846140 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:57:54.846140 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:57:54.846140 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 24 05:57:54.846140 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:57:54.846140 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:57:54.846140 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:57:54.846140 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:57:54.846140 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:57:54.846140 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:57:54.846140 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:57:54.846140 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:57:54.847208 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 05:57:54.847208 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 05:57:54.847208 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 05:57:54.847208 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 05:57:54.847208 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 05:57:54.847208 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 05:57:54.847208 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 05:57:54.847208 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 24 05:57:54.853235 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 340/684] 10.222.0.62 () {56 vars in 1111 bytes} [Wed Aug 24 05:57:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:57:54.853235 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b8add0-45c3-48e2-b586-d6e8029e98ba None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:54.896904 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:54.897407 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 345/685] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Aug 24 05:57:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 05:57:54.898931 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0840a1ca-50f2-411d-94a6-7e32e0f89f56 None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:54.954594 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b8add0-45c3-48e2-b586-d6e8029e98ba None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:54.993727 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6455ed-a175-4d98-a26b-aa7c9d989127 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:54.994498 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6455ed-a175-4d98-a26b-aa7c9d989127 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:54.994872 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6455ed-a175-4d98-a26b-aa7c9d989127 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:54.995239 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6455ed-a175-4d98-a26b-aa7c9d989127 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:55.022894 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6455ed-a175-4d98-a26b-aa7c9d989127 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:55.025202 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6455ed-a175-4d98-a26b-aa7c9d989127 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:55.033631 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:55.050616 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0840a1ca-50f2-411d-94a6-7e32e0f89f56 None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:55.057551 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-bc6455ed-a175-4d98-a26b-aa7c9d989127 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:57:55.057551 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:57:55.057551 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:57:55.057551 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:57:55.057551 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:57:55.057551 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:57:55.057551 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:57:55.057551 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:57:55.057551 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:57:55.057551 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:57:55.057551 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:57:55.057551 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:57:55.057551 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 05:57:55.057551 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 05:57:55.057551 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 05:57:55.057551 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 05:57:55.064939 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:57:55.064939 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:57:55.064939 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:57:55.064939 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:57:55.064939 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:57:55.064939 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:57:55.064939 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:57:55.064939 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 24 05:57:55.064939 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:57:55.064939 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:57:55.064939 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:57:55.064939 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:57:55.064939 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:57:55.064939 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:57:55.064939 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:57:55.064939 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:57:55.064939 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 05:57:55.065826 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 05:57:55.065826 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:57:55.065826 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 24 05:57:55.065826 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 341/686] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Aug 24 05:57:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:57:55.065826 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6455ed-a175-4d98-a26b-aa7c9d989127 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:55.089943 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2957c5b-622a-4463-b63a-dbba97a73627 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:55.091434 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2957c5b-622a-4463-b63a-dbba97a73627 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:55.091908 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2957c5b-622a-4463-b63a-dbba97a73627 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:55.092353 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2957c5b-622a-4463-b63a-dbba97a73627 None admin] PATH_INFO: `/v3/projects` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:55.093887 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2957c5b-622a-4463-b63a-dbba97a73627 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:57:55.094381 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2957c5b-622a-4463-b63a-dbba97a73627 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:55.096027 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2957c5b-622a-4463-b63a-dbba97a73627 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:55.108952 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 346/687] 10.222.0.62 () {56 vars in 1148 bytes} [Wed Aug 24 05:57:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:57:55.114851 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2957c5b-622a-4463-b63a-dbba97a73627 None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:55.230637 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:55.240752 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6455ed-a175-4d98-a26b-aa7c9d989127 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:55.285044 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f800ddbb-477a-48ff-8b6c-9d4b629368bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:55.285761 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f800ddbb-477a-48ff-8b6c-9d4b629368bc None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:55.286102 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f800ddbb-477a-48ff-8b6c-9d4b629368bc None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:55.286440 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f800ddbb-477a-48ff-8b6c-9d4b629368bc None admin] PATH_INFO: `/v3/domains` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:55.287142 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f800ddbb-477a-48ff-8b6c-9d4b629368bc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:57:55.287500 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f800ddbb-477a-48ff-8b6c-9d4b629368bc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:55.288965 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f800ddbb-477a-48ff-8b6c-9d4b629368bc None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:55.301222 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2957c5b-622a-4463-b63a-dbba97a73627 None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:55.302613 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 342/688] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Aug 24 05:57:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:57:55.304631 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f800ddbb-477a-48ff-8b6c-9d4b629368bc None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:55.339832 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab71f7c-76b5-45d8-847a-8ae459962a6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:55.340533 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab71f7c-76b5-45d8-847a-8ae459962a6c None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:55.340783 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab71f7c-76b5-45d8-847a-8ae459962a6c None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:55.341056 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab71f7c-76b5-45d8-847a-8ae459962a6c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:55.341839 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab71f7c-76b5-45d8-847a-8ae459962a6c None admin] RBAC: Adding query filter params (role.id=5d95a5fc0f324a92905533aa8d34ac15, scope.project.id=181f4762037e454489d9552cc603179c, user.id=0bb6c79db75e4d169331c02133d1ec32) {{(pid=81851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:57:55.342151 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab71f7c-76b5-45d8-847a-8ae459962a6c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:55.344588 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab71f7c-76b5-45d8-847a-8ae459962a6c None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:55.351428 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 347/689] 10.222.0.62 () {56 vars in 1374 bytes} [Wed Aug 24 05:57:55 2022] GET /identity/v3/role_assignments?user.id=0bb6c79db75e4d169331c02133d1ec32&scope.project.id=181f4762037e454489d9552cc603179c&role.id=5d95a5fc0f324a92905533aa8d34ac15 => generated 608 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:57:55.352245 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab71f7c-76b5-45d8-847a-8ae459962a6c None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:55.384621 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:55.508015 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f800ddbb-477a-48ff-8b6c-9d4b629368bc None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:55.526590 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab71f7c-76b5-45d8-847a-8ae459962a6c None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:55.549952 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a761c763-fe14-4f4b-81a5-14c65214a306 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:55.550597 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a761c763-fe14-4f4b-81a5-14c65214a306 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:55.550876 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a761c763-fe14-4f4b-81a5-14c65214a306 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:55.551146 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a761c763-fe14-4f4b-81a5-14c65214a306 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:55.564791 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a761c763-fe14-4f4b-81a5-14c65214a306 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:57:55.567375 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f195a6-6007-467f-b053-97ba3f59831f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:55.567998 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f195a6-6007-467f-b053-97ba3f59831f None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:55.568273 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f195a6-6007-467f-b053-97ba3f59831f None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:55.568609 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f195a6-6007-467f-b053-97ba3f59831f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:55.569890 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-a761c763-fe14-4f4b-81a5-14c65214a306 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 05:57:55.569890 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:57:55.569890 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:57:55.569890 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:57:55.569890 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:57:55.569890 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:57:55.569890 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:57:55.569890 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:57:55.569890 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:57:55.569890 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:57:55.569890 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:57:55.569890 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:57:55.569890 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:57:55.569890 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:57:55.569890 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:57:55.569890 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:57:55.570807 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:57:55.570807 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:57:55.570807 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:57:55.570807 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:57:55.570807 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:57:55.570807 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:57:55.570807 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:57:55.570807 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:57:55.570807 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:57:55.570807 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:57:55.570807 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:57:55.570807 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 24 05:57:55.570807 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:57:55.570807 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:57:55.570807 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:57:55.570807 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:57:55.571679 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:57:55.571679 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:57:55.571679 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:57:55.571679 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:57:55.571679 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:57:55.571679 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:57:55.571679 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:57:55.571679 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:57:55.571679 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:57:55.571679 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:57:55.571679 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 05:57:55.571679 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 24 05:57:55.571679 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 343/690] 10.222.0.62 () {56 vars in 1096 bytes} [Wed Aug 24 05:57:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:57:55.572910 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9274caa9-84d9-4469-b2bc-09deee5733f8 None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:55.572910 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9274caa9-84d9-4469-b2bc-09deee5733f8 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:55.573111 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9274caa9-84d9-4469-b2bc-09deee5733f8 None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:55.579017 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 344/691] 10.222.0.62 () {54 vars in 927 bytes} [Wed Aug 24 05:57:55 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:57:55.580152 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71184166-c807-450d-98b9-de5739b07032 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:55.580461 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71184166-c807-450d-98b9-de5739b07032 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:55.580736 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71184166-c807-450d-98b9-de5739b07032 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:55.582845 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f195a6-6007-467f-b053-97ba3f59831f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:55.584923 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f195a6-6007-467f-b053-97ba3f59831f None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:55.598428 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application [None req-20f195a6-6007-467f-b053-97ba3f59831f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 05:57:55.598428 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:57:55.598428 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:57:55.598428 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:57:55.598428 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:57:55.598428 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:57:55.598428 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:57:55.598428 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:57:55.598428 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:57:55.598428 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:57:55.598428 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:57:55.598428 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:57:55.598428 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 05:57:55.598428 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 05:57:55.598428 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 05:57:55.600669 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 05:57:55.600669 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:57:55.600669 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:57:55.600669 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:57:55.600669 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:57:55.600669 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:57:55.600669 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:57:55.600669 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:57:55.600669 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application with Lock( Aug 24 05:57:55.600669 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:57:55.600669 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:57:55.600669 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:57:55.600669 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:57:55.600669 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:57:55.600669 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:57:55.600669 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:57:55.600669 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:57:55.602806 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 05:57:55.602806 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 05:57:55.602806 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 05:57:55.602806 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 05:57:55.602806 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 05:57:55.602806 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 05:57:55.602806 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 05:57:55.602806 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Aug 24 05:57:55.603698 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-71184166-c807-450d-98b9-de5739b07032 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:57:55.603857 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 348/692] 10.222.0.62 () {56 vars in 1111 bytes} [Wed Aug 24 05:57:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:57:55.603857 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f195a6-6007-467f-b053-97ba3f59831f None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:55.604107 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-71184166-c807-450d-98b9-de5739b07032 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:57:55.609098 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:55.677525 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 345/693] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Aug 24 05:57:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 05:57:55.681412 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71184166-c807-450d-98b9-de5739b07032 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:55.684914 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:55.720525 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f195a6-6007-467f-b053-97ba3f59831f None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:55.756970 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed39ee65-3162-4445-bf48-70f00bb74dc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:55.757578 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed39ee65-3162-4445-bf48-70f00bb74dc9 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:55.757861 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed39ee65-3162-4445-bf48-70f00bb74dc9 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:55.758132 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed39ee65-3162-4445-bf48-70f00bb74dc9 None admin] PATH_INFO: `/v3/roles` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:55.758790 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed39ee65-3162-4445-bf48-70f00bb74dc9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:57:55.759091 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed39ee65-3162-4445-bf48-70f00bb74dc9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:55.760475 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed39ee65-3162-4445-bf48-70f00bb74dc9 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:55.770162 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 349/694] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Aug 24 05:57:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:57:55.776071 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed39ee65-3162-4445-bf48-70f00bb74dc9 None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:55.882194 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71184166-c807-450d-98b9-de5739b07032 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:55.926570 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22072983-39b2-4b26-bb1e-7cde34728314 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:55.927188 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22072983-39b2-4b26-bb1e-7cde34728314 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:55.927469 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22072983-39b2-4b26-bb1e-7cde34728314 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:55.927740 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22072983-39b2-4b26-bb1e-7cde34728314 None admin] PATH_INFO: `/v3/projects` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:55.928433 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22072983-39b2-4b26-bb1e-7cde34728314 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:57:55.928767 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22072983-39b2-4b26-bb1e-7cde34728314 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:55.936509 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22072983-39b2-4b26-bb1e-7cde34728314 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:55.943771 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 346/695] 10.222.0.62 () {56 vars in 1148 bytes} [Wed Aug 24 05:57:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:57:55.952516 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22072983-39b2-4b26-bb1e-7cde34728314 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:55.952891 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:55.983109 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed39ee65-3162-4445-bf48-70f00bb74dc9 None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:56.021863 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:56.027295 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20591aa-2cd7-4ec0-a254-bb93721ee42f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:56.028155 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20591aa-2cd7-4ec0-a254-bb93721ee42f None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:56.028517 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20591aa-2cd7-4ec0-a254-bb93721ee42f None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:56.031464 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20591aa-2cd7-4ec0-a254-bb93721ee42f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:56.040699 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20591aa-2cd7-4ec0-a254-bb93721ee42f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:56.042285 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20591aa-2cd7-4ec0-a254-bb93721ee42f None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:56.060519 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application [None req-a20591aa-2cd7-4ec0-a254-bb93721ee42f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:57:56.060519 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:57:56.060519 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:57:56.060519 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:57:56.060519 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:57:56.060519 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:57:56.060519 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:57:56.060519 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:57:56.060519 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:57:56.060519 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:57:56.060519 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:57:56.060519 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:57:56.060519 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 05:57:56.060519 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 05:57:56.060519 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 05:57:56.060519 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 05:57:56.061498 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:57:56.061498 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:57:56.061498 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:57:56.061498 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:57:56.061498 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:57:56.061498 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:57:56.061498 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:57:56.061498 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application with Lock( Aug 24 05:57:56.061498 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:57:56.061498 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:57:56.061498 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:57:56.061498 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:57:56.061498 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:57:56.061498 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:57:56.061498 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:57:56.061498 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:57:56.061498 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 05:57:56.063150 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 05:57:56.063150 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:57:56.063150 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Aug 24 05:57:56.063150 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 350/696] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Aug 24 05:57:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 05:57:56.063384 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20591aa-2cd7-4ec0-a254-bb93721ee42f None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:56.095170 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22072983-39b2-4b26-bb1e-7cde34728314 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:56.135662 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69878e18-a0f7-45ad-ac71-2a22189c4431 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:56.136237 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69878e18-a0f7-45ad-ac71-2a22189c4431 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:56.136538 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69878e18-a0f7-45ad-ac71-2a22189c4431 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:56.136867 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69878e18-a0f7-45ad-ac71-2a22189c4431 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:56.137592 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69878e18-a0f7-45ad-ac71-2a22189c4431 None admin] RBAC: Adding query filter params (role.id=5d95a5fc0f324a92905533aa8d34ac15, scope.project.id=181f4762037e454489d9552cc603179c, user.id=14d4fed04fb144d08918bd5d4f808d23) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:57:56.137899 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69878e18-a0f7-45ad-ac71-2a22189c4431 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:56.139613 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69878e18-a0f7-45ad-ac71-2a22189c4431 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:56.146644 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 347/697] 10.222.0.62 () {56 vars in 1374 bytes} [Wed Aug 24 05:57:55 2022] GET /identity/v3/role_assignments?user.id=14d4fed04fb144d08918bd5d4f808d23&scope.project.id=181f4762037e454489d9552cc603179c&role.id=5d95a5fc0f324a92905533aa8d34ac15 => generated 608 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:57:56.147521 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69878e18-a0f7-45ad-ac71-2a22189c4431 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:56.189818 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20591aa-2cd7-4ec0-a254-bb93721ee42f None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:56.225736 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:56.268581 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69878e18-a0f7-45ad-ac71-2a22189c4431 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:56.290815 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd41b3f-bbc2-4876-8dba-8e3d77b4b401 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:56.291468 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd41b3f-bbc2-4876-8dba-8e3d77b4b401 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:56.291750 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd41b3f-bbc2-4876-8dba-8e3d77b4b401 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:56.292022 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd41b3f-bbc2-4876-8dba-8e3d77b4b401 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:56.308987 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e7f7ad-e29f-491c-b435-53d7e4a0a17d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:56.309970 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e7f7ad-e29f-491c-b435-53d7e4a0a17d None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:56.310501 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e7f7ad-e29f-491c-b435-53d7e4a0a17d None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:56.310959 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e7f7ad-e29f-491c-b435-53d7e4a0a17d None admin] PATH_INFO: `/v3/domains` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:56.312379 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e7f7ad-e29f-491c-b435-53d7e4a0a17d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:57:56.312924 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e7f7ad-e29f-491c-b435-53d7e4a0a17d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:56.315341 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e7f7ad-e29f-491c-b435-53d7e4a0a17d None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:56.324541 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd41b3f-bbc2-4876-8dba-8e3d77b4b401 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:56.325943 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd41b3f-bbc2-4876-8dba-8e3d77b4b401 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:56.341382 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 348/698] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Aug 24 05:57:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:57:56.346631 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e7f7ad-e29f-491c-b435-53d7e4a0a17d None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:56.346984 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application [None req-9dd41b3f-bbc2-4876-8dba-8e3d77b4b401 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:57:56.346984 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:57:56.346984 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:57:56.346984 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:57:56.346984 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:57:56.346984 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:57:56.346984 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:57:56.346984 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:57:56.346984 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:57:56.346984 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:57:56.346984 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:57:56.346984 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:57:56.346984 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 05:57:56.346984 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 05:57:56.346984 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 05:57:56.346984 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 05:57:56.347885 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:57:56.347885 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:57:56.347885 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:57:56.347885 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:57:56.347885 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:57:56.347885 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:57:56.347885 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:57:56.347885 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application with Lock( Aug 24 05:57:56.347885 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:57:56.347885 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:57:56.347885 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:57:56.347885 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:57:56.347885 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:57:56.347885 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:57:56.347885 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:57:56.347885 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:57:56.347885 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 05:57:56.349029 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 05:57:56.349029 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:57:56.349029 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Aug 24 05:57:56.349029 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 351/699] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Aug 24 05:57:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:57:56.353698 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd41b3f-bbc2-4876-8dba-8e3d77b4b401 None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:56.394505 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:56.457161 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e7f7ad-e29f-491c-b435-53d7e4a0a17d None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:56.465411 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd41b3f-bbc2-4876-8dba-8e3d77b4b401 None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:56.488832 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:56.501797 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431f5506-358c-4ffe-93e8-ac2af876a390 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:56.504591 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431f5506-358c-4ffe-93e8-ac2af876a390 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:56.504591 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431f5506-358c-4ffe-93e8-ac2af876a390 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:56.504591 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431f5506-358c-4ffe-93e8-ac2af876a390 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:56.510899 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4434de7b-e47b-4480-9cea-29feb1a7cefd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:56.511231 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431f5506-358c-4ffe-93e8-ac2af876a390 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:56.511943 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4434de7b-e47b-4480-9cea-29feb1a7cefd None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:56.512218 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4434de7b-e47b-4480-9cea-29feb1a7cefd None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:56.512486 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4434de7b-e47b-4480-9cea-29feb1a7cefd None admin] PATH_INFO: `/v3/domains` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:56.512810 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431f5506-358c-4ffe-93e8-ac2af876a390 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:56.513213 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4434de7b-e47b-4480-9cea-29feb1a7cefd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:57:56.513444 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4434de7b-e47b-4480-9cea-29feb1a7cefd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:56.514809 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4434de7b-e47b-4480-9cea-29feb1a7cefd None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:56.528397 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 352/700] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Aug 24 05:57:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:57:56.529330 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-431f5506-358c-4ffe-93e8-ac2af876a390 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 05:57:56.529330 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:57:56.529330 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:57:56.529330 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:57:56.529330 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:57:56.529330 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:57:56.529330 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:57:56.529330 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:57:56.529330 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:57:56.529330 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:57:56.529330 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:57:56.529330 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:57:56.529330 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 05:57:56.529330 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 05:57:56.529330 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 05:57:56.529330 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 05:57:56.530263 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:57:56.530263 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:57:56.530263 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 05:57:56.530263 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 05:57:56.530263 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 05:57:56.530263 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 05:57:56.530263 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:57:56.530263 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:57:56.530263 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:57:56.530263 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:57:56.530263 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:57:56.530263 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 24 05:57:56.530263 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:57:56.530263 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:57:56.530263 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:57:56.530263 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:57:56.530263 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:57:56.530263 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:57:56.531235 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:57:56.531235 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:57:56.531235 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 05:57:56.531235 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 05:57:56.531235 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 05:57:56.531235 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 05:57:56.531235 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 05:57:56.531235 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 05:57:56.531235 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 05:57:56.531235 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 24 05:57:56.533346 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61173c5f-c80f-4bb0-9996-86cefb259f1d None None] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:56.533689 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 349/701] 10.222.0.62 () {56 vars in 1093 bytes} [Wed Aug 24 05:57:56 2022] GET /identity/v3/users/nova => generated 82 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 05:57:56.534406 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61173c5f-c80f-4bb0-9996-86cefb259f1d None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:56.534756 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431f5506-358c-4ffe-93e8-ac2af876a390 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:56.535373 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61173c5f-c80f-4bb0-9996-86cefb259f1d None None] PATH_INFO: `/` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:56.537193 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 353/702] 10.222.0.62 () {54 vars in 927 bytes} [Wed Aug 24 05:57:56 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:57:56.546559 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb6106b-6964-4576-b6ac-a780c45a9d28 None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:56.546559 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb6106b-6964-4576-b6ac-a780c45a9d28 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:56.546559 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb6106b-6964-4576-b6ac-a780c45a9d28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:56.568256 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-afb6106b-6964-4576-b6ac-a780c45a9d28 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:57:56.569725 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-afb6106b-6964-4576-b6ac-a780c45a9d28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:57:56.644848 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431f5506-358c-4ffe-93e8-ac2af876a390 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:56.660033 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 354/703] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Aug 24 05:57:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 05:57:56.660958 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb6106b-6964-4576-b6ac-a780c45a9d28 None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:56.680320 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32f1b88-67f6-4850-bce3-3cf1f97572df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:56.680999 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32f1b88-67f6-4850-bce3-3cf1f97572df None admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:56.681257 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32f1b88-67f6-4850-bce3-3cf1f97572df None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:56.681534 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32f1b88-67f6-4850-bce3-3cf1f97572df None admin] PATH_INFO: `/v3/users` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:56.682178 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32f1b88-67f6-4850-bce3-3cf1f97572df None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:56.683633 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32f1b88-67f6-4850-bce3-3cf1f97572df None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:56.704099 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:56.779059 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 350/704] 10.222.0.62 () {60 vars in 1132 bytes} [Wed Aug 24 05:57:56 2022] POST /identity/v3/users => generated 254 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:57:56.784432 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb6106b-6964-4576-b6ac-a780c45a9d28 None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:56.833106 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7befd933-9995-47b7-aafe-1a7c27dd5ff6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:56.833718 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7befd933-9995-47b7-aafe-1a7c27dd5ff6 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:56.833994 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7befd933-9995-47b7-aafe-1a7c27dd5ff6 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:56.834265 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7befd933-9995-47b7-aafe-1a7c27dd5ff6 None admin] PATH_INFO: `/v3/users` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:56.834994 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7befd933-9995-47b7-aafe-1a7c27dd5ff6 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:57:56.835301 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7befd933-9995-47b7-aafe-1a7c27dd5ff6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:56.836851 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7befd933-9995-47b7-aafe-1a7c27dd5ff6 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:56.860439 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 355/705] 10.222.0.62 () {56 vars in 1133 bytes} [Wed Aug 24 05:57:56 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:57:56.862580 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32f1b88-67f6-4850-bce3-3cf1f97572df None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:56.931985 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:56.993184 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32f1b88-67f6-4850-bce3-3cf1f97572df None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:57.030200 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3a266a-c129-4948-8264-5be0657f24d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:57.030853 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3a266a-c129-4948-8264-5be0657f24d8 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:57.031137 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3a266a-c129-4948-8264-5be0657f24d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:57.031410 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3a266a-c129-4948-8264-5be0657f24d8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:57.039328 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7befd933-9995-47b7-aafe-1a7c27dd5ff6 None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:57.104041 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3a266a-c129-4948-8264-5be0657f24d8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:57.105591 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3a266a-c129-4948-8264-5be0657f24d8 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:57.149390 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:57.160943 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:57.174213 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7befd933-9995-47b7-aafe-1a7c27dd5ff6 None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:57.212088 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67f6c37-3d5d-42ae-a2b8-339860bf2b2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:57.212668 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67f6c37-3d5d-42ae-a2b8-339860bf2b2b None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:57.212948 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67f6c37-3d5d-42ae-a2b8-339860bf2b2b None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:57.213218 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67f6c37-3d5d-42ae-a2b8-339860bf2b2b None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:57.213790 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67f6c37-3d5d-42ae-a2b8-339860bf2b2b None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:57.214837 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 351/706] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Aug 24 05:57:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 05:57:57.217127 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67f6c37-3d5d-42ae-a2b8-339860bf2b2b None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:57.220546 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3a266a-c129-4948-8264-5be0657f24d8 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:57.233132 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application [None req-b67f6c37-3d5d-42ae-a2b8-339860bf2b2b None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 05:57:57.233132 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:57:57.233132 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 05:57:57.233132 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 05:57:57.233132 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 05:57:57.233132 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 05:57:57.233132 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 05:57:57.233132 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 05:57:57.233132 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 05:57:57.233132 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Aug 24 05:57:57.233132 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 05:57:57.233132 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Aug 24 05:57:57.233132 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:57:57.233132 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:57:57.233132 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:57:57.233132 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:57:57.233132 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:57:57.234230 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:57:57.234230 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:57:57.234230 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:57:57.234230 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:57:57.234230 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:57:57.234230 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:57:57.234230 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 05:57:57.234230 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 05:57:57.234230 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 05:57:57.234230 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 05:57:57.234230 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:57:57.234230 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:57:57.234230 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:57:57.234230 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:57:57.234230 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:57:57.234230 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:57:57.235068 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:57:57.235068 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application with Lock( Aug 24 05:57:57.235068 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:57:57.235068 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:57:57.235068 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:57:57.235068 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:57:57.235068 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:57:57.235068 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:57:57.235068 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:57:57.235068 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:57:57.235068 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 05:57:57.235068 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 05:57:57.235068 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 05:57:57.235068 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Aug 24 05:57:57.239155 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 356/707] 10.222.0.62 () {56 vars in 1129 bytes} [Wed Aug 24 05:57:57 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:57:57.241302 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67f6c37-3d5d-42ae-a2b8-339860bf2b2b None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:57.326332 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3a266a-c129-4948-8264-5be0657f24d8 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:57.353328 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67f6c37-3d5d-42ae-a2b8-339860bf2b2b None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:57.377058 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d0b81e-09a5-457c-9f83-f15d8e4565fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:57.378165 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d0b81e-09a5-457c-9f83-f15d8e4565fe None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:57.378439 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d0b81e-09a5-457c-9f83-f15d8e4565fe None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:57.378977 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d0b81e-09a5-457c-9f83-f15d8e4565fe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:57.391895 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d0b81e-09a5-457c-9f83-f15d8e4565fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:57.393336 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae57a7f-5107-46ad-b4b3-2032d6c438ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:57.393468 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d0b81e-09a5-457c-9f83-f15d8e4565fe None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:57.398820 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae57a7f-5107-46ad-b4b3-2032d6c438ab None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:57.398820 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae57a7f-5107-46ad-b4b3-2032d6c438ab None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:57.398820 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae57a7f-5107-46ad-b4b3-2032d6c438ab None admin] PATH_INFO: `/v3/services` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:57.399067 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:57.399139 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae57a7f-5107-46ad-b4b3-2032d6c438ab None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:57:57.399139 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae57a7f-5107-46ad-b4b3-2032d6c438ab None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:57.399139 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae57a7f-5107-46ad-b4b3-2032d6c438ab None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:57.404626 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 357/708] 10.222.0.62 () {56 vars in 1124 bytes} [Wed Aug 24 05:57:57 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:57:57.408745 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae57a7f-5107-46ad-b4b3-2032d6c438ab None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:57.409429 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-e2d0b81e-09a5-457c-9f83-f15d8e4565fe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:57:57.409429 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:57:57.409429 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:57:57.409429 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:57:57.409429 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:57:57.409429 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:57:57.409429 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:57:57.409429 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:57:57.409429 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:57:57.409429 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:57:57.409429 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:57:57.409429 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:57:57.409429 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 05:57:57.409429 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 05:57:57.409429 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 05:57:57.409429 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 05:57:57.410337 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:57:57.410337 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:57:57.410337 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:57:57.410337 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:57:57.410337 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:57:57.410337 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:57:57.410337 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:57:57.410337 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 24 05:57:57.410337 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:57:57.410337 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:57:57.410337 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:57:57.410337 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:57:57.410337 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:57:57.410337 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:57:57.410337 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:57:57.410337 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:57:57.410337 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 05:57:57.411223 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 05:57:57.411223 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:57:57.411223 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 24 05:57:57.411223 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 352/709] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Aug 24 05:57:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:57:57.426399 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d0b81e-09a5-457c-9f83-f15d8e4565fe None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:57.434510 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:57.567343 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae57a7f-5107-46ad-b4b3-2032d6c438ab None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:57.606186 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7bc2bb-d2ce-4968-bede-88c7aff05024 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:57.606815 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7bc2bb-d2ce-4968-bede-88c7aff05024 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:57.607094 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7bc2bb-d2ce-4968-bede-88c7aff05024 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:57.607366 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7bc2bb-d2ce-4968-bede-88c7aff05024 None admin] PATH_INFO: `/v3/services` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:57.608016 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7bc2bb-d2ce-4968-bede-88c7aff05024 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:57:57.608316 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7bc2bb-d2ce-4968-bede-88c7aff05024 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:57.610344 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7bc2bb-d2ce-4968-bede-88c7aff05024 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:57.623341 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 358/710] 10.222.0.62 () {56 vars in 1124 bytes} [Wed Aug 24 05:57:57 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:57:57.650276 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e81ce76-20fb-46c5-b674-4fd203bf5d68 None None] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:57.650899 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e81ce76-20fb-46c5-b674-4fd203bf5d68 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:57.651395 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e81ce76-20fb-46c5-b674-4fd203bf5d68 None None] PATH_INFO: `/` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:57.651598 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d0b81e-09a5-457c-9f83-f15d8e4565fe None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:57.653537 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 359/711] 10.222.0.62 () {54 vars in 927 bytes} [Wed Aug 24 05:57:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 05:57:57.664282 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08e6baf-b258-409c-af02-786d062102f4 None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:57.664955 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08e6baf-b258-409c-af02-786d062102f4 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:57.665431 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08e6baf-b258-409c-af02-786d062102f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:57.700896 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-b08e6baf-b258-409c-af02-786d062102f4 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:57:57.704094 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-b08e6baf-b258-409c-af02-786d062102f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:57:57.724939 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6551c1c4-da11-431d-a5bf-e0b0d35c7bed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:57.726316 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6551c1c4-da11-431d-a5bf-e0b0d35c7bed None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:57.726805 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6551c1c4-da11-431d-a5bf-e0b0d35c7bed None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:57.727320 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6551c1c4-da11-431d-a5bf-e0b0d35c7bed None admin] PATH_INFO: `/v3/domains` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:57.728244 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6551c1c4-da11-431d-a5bf-e0b0d35c7bed None admin] RBAC: Adding query filter params (name=Default) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:57:57.732988 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6551c1c4-da11-431d-a5bf-e0b0d35c7bed None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:57.734972 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6551c1c4-da11-431d-a5bf-e0b0d35c7bed None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:57.759094 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 353/712] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Aug 24 05:57:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:57:57.765652 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6551c1c4-da11-431d-a5bf-e0b0d35c7bed None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:57.798985 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 360/713] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Aug 24 05:57:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 05:57:57.851115 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:57.903638 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6551c1c4-da11-431d-a5bf-e0b0d35c7bed None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:57.940264 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c44cf6-58a8-4eeb-afcd-4dbe63845ea6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:57.941324 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c44cf6-58a8-4eeb-afcd-4dbe63845ea6 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:57.941808 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c44cf6-58a8-4eeb-afcd-4dbe63845ea6 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:57.942277 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c44cf6-58a8-4eeb-afcd-4dbe63845ea6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:57.949569 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c44cf6-58a8-4eeb-afcd-4dbe63845ea6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:57.951244 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c44cf6-58a8-4eeb-afcd-4dbe63845ea6 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:57.962678 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-c9c44cf6-58a8-4eeb-afcd-4dbe63845ea6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 05:57:57.962678 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:57:57.962678 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:57:57.962678 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:57:57.962678 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:57:57.962678 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:57:57.962678 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:57:57.962678 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:57:57.962678 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:57:57.962678 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:57:57.962678 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:57:57.962678 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:57:57.962678 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 05:57:57.962678 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 05:57:57.962678 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 05:57:57.963615 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 05:57:57.963615 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:57:57.963615 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:57:57.963615 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:57:57.963615 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:57:57.963615 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:57:57.963615 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:57:57.963615 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:57:57.963615 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 24 05:57:57.963615 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:57:57.963615 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:57:57.963615 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:57:57.963615 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:57:57.963615 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:57:57.963615 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:57:57.963615 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:57:57.963615 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:57:57.964696 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 05:57:57.964696 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 05:57:57.964696 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 05:57:57.964696 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 05:57:57.964696 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 05:57:57.964696 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 05:57:57.964696 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 05:57:57.964696 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 24 05:57:57.967106 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 354/714] 10.222.0.62 () {56 vars in 1111 bytes} [Wed Aug 24 05:57:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:57:57.973128 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08e6baf-b258-409c-af02-786d062102f4 None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:57.982258 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:58.085514 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08e6baf-b258-409c-af02-786d062102f4 None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:58.119536 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b8aecd-6c17-4696-aa2b-739a2aa517aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:58.120060 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b8aecd-6c17-4696-aa2b-739a2aa517aa None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:58.120336 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b8aecd-6c17-4696-aa2b-739a2aa517aa None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:58.120633 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b8aecd-6c17-4696-aa2b-739a2aa517aa None admin] PATH_INFO: `/v3/projects` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:58.121327 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b8aecd-6c17-4696-aa2b-739a2aa517aa None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:57:58.121628 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b8aecd-6c17-4696-aa2b-739a2aa517aa None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:58.123135 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b8aecd-6c17-4696-aa2b-739a2aa517aa None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:58.135078 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 361/715] 10.222.0.62 () {56 vars in 1148 bytes} [Wed Aug 24 05:57:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:57:58.141231 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c44cf6-58a8-4eeb-afcd-4dbe63845ea6 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:58.169702 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b8aecd-6c17-4696-aa2b-739a2aa517aa None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:58.316253 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:58.344044 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b8aecd-6c17-4696-aa2b-739a2aa517aa None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:58.345438 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c44cf6-58a8-4eeb-afcd-4dbe63845ea6 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:58.376587 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:58.382616 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad763e6-2416-4c27-b7d5-1a382e3488c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:58.383317 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad763e6-2416-4c27-b7d5-1a382e3488c9 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:58.383596 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad763e6-2416-4c27-b7d5-1a382e3488c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:58.383874 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad763e6-2416-4c27-b7d5-1a382e3488c9 None admin] PATH_INFO: `/v3/services/network` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:58.385779 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad763e6-2416-4c27-b7d5-1a382e3488c9 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:58.388044 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad763e6-2416-4c27-b7d5-1a382e3488c9 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:58.400369 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application [None req-7ad763e6-2416-4c27-b7d5-1a382e3488c9 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 05:57:58.400369 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:57:58.400369 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 05:57:58.400369 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 05:57:58.400369 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 05:57:58.400369 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 05:57:58.400369 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 05:57:58.400369 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 05:57:58.400369 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 05:57:58.400369 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Aug 24 05:57:58.400369 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 05:57:58.400369 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Aug 24 05:57:58.400369 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:57:58.400369 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:57:58.400369 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:57:58.400369 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:57:58.400369 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:57:58.401514 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:57:58.401514 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:57:58.401514 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:57:58.401514 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:57:58.401514 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:57:58.401514 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:57:58.401514 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 05:57:58.401514 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 05:57:58.401514 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 05:57:58.401514 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 05:57:58.401514 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:57:58.401514 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:57:58.401514 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:57:58.401514 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:57:58.401514 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:57:58.401514 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:57:58.402368 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:57:58.402368 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application with Lock( Aug 24 05:57:58.402368 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:57:58.402368 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:57:58.402368 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:57:58.402368 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:57:58.402368 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:57:58.402368 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:57:58.402368 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:57:58.402368 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:57:58.402368 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 05:57:58.402368 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 05:57:58.402368 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 05:57:58.402368 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Aug 24 05:57:58.402368 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 362/716] 10.222.0.62 () {56 vars in 1111 bytes} [Wed Aug 24 05:57:58 2022] GET /identity/v3/services/network => generated 88 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 05:57:58.411422 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad763e6-2416-4c27-b7d5-1a382e3488c9 None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:58.423850 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfe34cb-7b10-4a88-b2ae-58ada14ca96b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:58.424460 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfe34cb-7b10-4a88-b2ae-58ada14ca96b None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:58.424761 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfe34cb-7b10-4a88-b2ae-58ada14ca96b None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:58.425031 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfe34cb-7b10-4a88-b2ae-58ada14ca96b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:58.425751 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfe34cb-7b10-4a88-b2ae-58ada14ca96b None admin] RBAC: Adding query filter params (role.id=49e47cd157d743f6905f1db07ec878a7, scope.project.id=181f4762037e454489d9552cc603179c, user.id=fb6d3743340241f98cacb9e0d063298c) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:57:58.426054 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfe34cb-7b10-4a88-b2ae-58ada14ca96b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:58.427484 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfe34cb-7b10-4a88-b2ae-58ada14ca96b None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:58.447556 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 355/717] 10.222.0.62 () {56 vars in 1374 bytes} [Wed Aug 24 05:57:58 2022] GET /identity/v3/role_assignments?user.id=fb6d3743340241f98cacb9e0d063298c&scope.project.id=181f4762037e454489d9552cc603179c&role.id=49e47cd157d743f6905f1db07ec878a7 => generated 259 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:57:58.554108 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1560185-8a12-4bd7-b79c-8f346a769509 None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:58.554406 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1560185-8a12-4bd7-b79c-8f346a769509 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:58.554652 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1560185-8a12-4bd7-b79c-8f346a769509 None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:58.556320 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 356/718] 10.222.0.62 () {54 vars in 927 bytes} [Wed Aug 24 05:57:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:57:58.562109 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad763e6-2416-4c27-b7d5-1a382e3488c9 None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:58.578554 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d30514-cdf3-49cf-9c95-589b63da9d92 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:58.578893 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d30514-cdf3-49cf-9c95-589b63da9d92 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:58.579101 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d30514-cdf3-49cf-9c95-589b63da9d92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:58.600096 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:58.609868 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-f9d30514-cdf3-49cf-9c95-589b63da9d92 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:57:58.611089 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-f9d30514-cdf3-49cf-9c95-589b63da9d92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:57:58.615224 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2b2963-28c6-4a1c-b9b4-f10f5f642628 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:58.615845 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2b2963-28c6-4a1c-b9b4-f10f5f642628 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:58.616121 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2b2963-28c6-4a1c-b9b4-f10f5f642628 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:58.616390 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2b2963-28c6-4a1c-b9b4-f10f5f642628 None admin] PATH_INFO: `/v3/services` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:58.617059 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2b2963-28c6-4a1c-b9b4-f10f5f642628 None admin] RBAC: Adding query filter params (name=network) {{(pid=81851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:57:58.617362 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2b2963-28c6-4a1c-b9b4-f10f5f642628 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:58.622852 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2b2963-28c6-4a1c-b9b4-f10f5f642628 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:58.630441 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 363/719] 10.222.0.62 () {56 vars in 1112 bytes} [Wed Aug 24 05:57:58 2022] GET /identity/v3/services?name=network => generated 124 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:57:58.631308 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2b2963-28c6-4a1c-b9b4-f10f5f642628 None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:58.729548 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 357/720] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Aug 24 05:57:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 05:57:58.753505 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2b2963-28c6-4a1c-b9b4-f10f5f642628 None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:58.805442 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ebf295-8178-4e15-a0d9-222131a80616 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:58.806065 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ebf295-8178-4e15-a0d9-222131a80616 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:58.806345 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ebf295-8178-4e15-a0d9-222131a80616 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:58.806616 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ebf295-8178-4e15-a0d9-222131a80616 None admin] PATH_INFO: `/v3/services` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:58.807249 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ebf295-8178-4e15-a0d9-222131a80616 None admin] RBAC: Adding query filter params (type=network) {{(pid=81851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:57:58.807550 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ebf295-8178-4e15-a0d9-222131a80616 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:58.808965 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ebf295-8178-4e15-a0d9-222131a80616 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:58.816404 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 364/721] 10.222.0.62 () {56 vars in 1112 bytes} [Wed Aug 24 05:57:58 2022] GET /identity/v3/services?type=network => generated 124 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:57:59.079870 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d30514-cdf3-49cf-9c95-589b63da9d92 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:59.189537 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d30514-cdf3-49cf-9c95-589b63da9d92 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:59.203460 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:59.246164 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2758a1-590d-4ec7-815d-183aeefb58ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:59.246863 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2758a1-590d-4ec7-815d-183aeefb58ec None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:59.247225 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2758a1-590d-4ec7-815d-183aeefb58ec None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:59.247501 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2758a1-590d-4ec7-815d-183aeefb58ec None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:59.265918 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2758a1-590d-4ec7-815d-183aeefb58ec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:57:59.270901 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-8e2758a1-590d-4ec7-815d-183aeefb58ec None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 05:57:59.270901 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:57:59.270901 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:57:59.270901 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:57:59.270901 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:57:59.270901 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:57:59.270901 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:57:59.270901 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:57:59.270901 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:57:59.270901 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:57:59.270901 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:57:59.270901 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:57:59.270901 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:57:59.270901 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:57:59.270901 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:57:59.270901 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:57:59.271802 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:57:59.271802 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:57:59.271802 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:57:59.271802 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:57:59.271802 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:57:59.271802 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:57:59.271802 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:57:59.271802 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:57:59.271802 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:57:59.271802 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:57:59.271802 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:57:59.271802 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 24 05:57:59.271802 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:57:59.271802 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:57:59.271802 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:57:59.271802 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:57:59.272685 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:57:59.272685 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:57:59.272685 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:57:59.272685 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:57:59.272685 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:57:59.272685 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:57:59.272685 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:57:59.272685 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:57:59.272685 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:57:59.272685 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:57:59.272685 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 05:57:59.272685 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 24 05:57:59.274064 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 358/722] 10.222.0.62 () {56 vars in 1096 bytes} [Wed Aug 24 05:57:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:57:59.279736 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ebf295-8178-4e15-a0d9-222131a80616 None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:59.366444 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:59.416760 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ebf295-8178-4e15-a0d9-222131a80616 None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:59.464011 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8b87da-2041-4f2b-b3ba-eb4cd7e9085c None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:59.464338 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8b87da-2041-4f2b-b3ba-eb4cd7e9085c None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:59.464574 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8b87da-2041-4f2b-b3ba-eb4cd7e9085c None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:59.467071 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 359/723] 10.222.0.62 () {54 vars in 927 bytes} [Wed Aug 24 05:57:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 05:57:59.470170 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db887ec-056c-4464-8e17-6111b803fd5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:59.470837 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db887ec-056c-4464-8e17-6111b803fd5a None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:59.471087 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93ea2f8-f30f-4f58-961c-5f113b8f9841 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:59.471191 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db887ec-056c-4464-8e17-6111b803fd5a None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:59.471368 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93ea2f8-f30f-4f58-961c-5f113b8f9841 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:59.471476 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db887ec-056c-4464-8e17-6111b803fd5a None admin] PATH_INFO: `/v3/roles` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:59.471649 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93ea2f8-f30f-4f58-961c-5f113b8f9841 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:59.472027 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db887ec-056c-4464-8e17-6111b803fd5a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:57:59.472329 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db887ec-056c-4464-8e17-6111b803fd5a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:59.473714 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db887ec-056c-4464-8e17-6111b803fd5a None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:59.492580 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 365/724] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Aug 24 05:57:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:57:59.497760 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db887ec-056c-4464-8e17-6111b803fd5a None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:59.501775 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-a93ea2f8-f30f-4f58-961c-5f113b8f9841 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:57:59.502958 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-a93ea2f8-f30f-4f58-961c-5f113b8f9841 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:57:59.598999 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 360/725] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Aug 24 05:57:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 05:57:59.642896 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db887ec-056c-4464-8e17-6111b803fd5a None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:59.663191 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:59.678248 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c690b5-677f-4ff0-ad34-5002f6f831d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:59.678879 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c690b5-677f-4ff0-ad34-5002f6f831d7 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:59.679159 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c690b5-677f-4ff0-ad34-5002f6f831d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:59.679429 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c690b5-677f-4ff0-ad34-5002f6f831d7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:59.690993 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c690b5-677f-4ff0-ad34-5002f6f831d7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:59.692406 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c690b5-677f-4ff0-ad34-5002f6f831d7 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:59.706782 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application [None req-c3c690b5-677f-4ff0-ad34-5002f6f831d7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:57:59.706782 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:57:59.706782 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:57:59.706782 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:57:59.706782 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:57:59.706782 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:57:59.706782 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:57:59.706782 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:57:59.706782 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:57:59.706782 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:57:59.706782 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:57:59.706782 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:57:59.706782 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 05:57:59.706782 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 05:57:59.706782 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 05:57:59.706782 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 05:57:59.707825 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:57:59.707825 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:57:59.707825 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:57:59.707825 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:57:59.707825 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:57:59.707825 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:57:59.707825 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:57:59.707825 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application with Lock( Aug 24 05:57:59.707825 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:57:59.707825 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:57:59.707825 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:57:59.707825 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:57:59.707825 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:57:59.707825 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:57:59.707825 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:57:59.707825 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:57:59.707825 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 05:57:59.708739 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 05:57:59.708739 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:57:59.708739 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Aug 24 05:57:59.708739 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 366/726] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Aug 24 05:57:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:57:59.717557 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93ea2f8-f30f-4f58-961c-5f113b8f9841 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:59.843186 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:59.846997 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93ea2f8-f30f-4f58-961c-5f113b8f9841 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:59.896081 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c32e37-0579-4260-b75a-77b4c659a949 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:59.896651 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c32e37-0579-4260-b75a-77b4c659a949 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:59.896932 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c32e37-0579-4260-b75a-77b4c659a949 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:59.897203 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c32e37-0579-4260-b75a-77b4c659a949 None admin] PATH_INFO: `/v3/domains` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:59.897836 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c32e37-0579-4260-b75a-77b4c659a949 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:57:59.898139 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c32e37-0579-4260-b75a-77b4c659a949 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:59.899500 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c32e37-0579-4260-b75a-77b4c659a949 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:59.915298 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 361/727] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Aug 24 05:57:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:57:59.921855 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c690b5-677f-4ff0-ad34-5002f6f831d7 None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:59.945921 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c32e37-0579-4260-b75a-77b4c659a949 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:00.100197 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c690b5-677f-4ff0-ad34-5002f6f831d7 None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:00.125240 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c32e37-0579-4260-b75a-77b4c659a949 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:00.165954 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f55c19b-ebb5-4b24-8b01-95eb460b7a16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:00.167380 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f55c19b-ebb5-4b24-8b01-95eb460b7a16 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:00.168395 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f55c19b-ebb5-4b24-8b01-95eb460b7a16 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:00.169109 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f55c19b-ebb5-4b24-8b01-95eb460b7a16 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:00.181623 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f55c19b-ebb5-4b24-8b01-95eb460b7a16 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:00.181623 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f55c19b-ebb5-4b24-8b01-95eb460b7a16 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:00.198595 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application [None req-3f55c19b-ebb5-4b24-8b01-95eb460b7a16 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 24 05:58:00.198595 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:58:00.198595 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:58:00.198595 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:58:00.198595 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:58:00.198595 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:58:00.198595 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:58:00.198595 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:58:00.198595 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:58:00.198595 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:58:00.198595 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:58:00.198595 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:58:00.198595 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 05:58:00.198595 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 05:58:00.198595 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 05:58:00.198595 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 05:58:00.199568 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:58:00.199568 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:58:00.199568 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 05:58:00.199568 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 05:58:00.199568 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 05:58:00.199568 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 05:58:00.199568 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:58:00.199568 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:58:00.199568 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:58:00.199568 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:58:00.199568 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:58:00.199568 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application with Lock( Aug 24 05:58:00.199568 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:58:00.199568 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:58:00.199568 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:58:00.199568 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:58:00.199568 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:58:00.199568 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:58:00.200482 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:58:00.200482 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:58:00.200482 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 05:58:00.200482 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 05:58:00.200482 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 05:58:00.200482 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 05:58:00.200482 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 05:58:00.200482 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 05:58:00.200482 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 24 05:58:00.200482 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Aug 24 05:58:00.201735 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 367/728] 10.222.0.62 () {56 vars in 1096 bytes} [Wed Aug 24 05:57:59 2022] GET /identity/v3/users/swift => generated 83 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:58:00.208217 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f55c19b-ebb5-4b24-8b01-95eb460b7a16 None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:00.209526 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d0d4e1-10df-48c7-8d9d-15182894a671 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:00.210393 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d0d4e1-10df-48c7-8d9d-15182894a671 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:00.210864 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d0d4e1-10df-48c7-8d9d-15182894a671 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:00.211457 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d0d4e1-10df-48c7-8d9d-15182894a671 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:00.230498 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d0d4e1-10df-48c7-8d9d-15182894a671 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:58:00.238176 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-32d0d4e1-10df-48c7-8d9d-15182894a671 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 05:58:00.238176 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:58:00.238176 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:58:00.238176 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:58:00.238176 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:58:00.238176 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:58:00.238176 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:58:00.238176 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:58:00.238176 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:58:00.238176 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:58:00.238176 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:58:00.238176 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:58:00.238176 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:58:00.238176 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:58:00.238176 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:58:00.238176 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:58:00.239391 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:58:00.239391 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:58:00.239391 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:58:00.239391 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:58:00.239391 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:58:00.239391 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:58:00.239391 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:58:00.239391 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:58:00.239391 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:58:00.239391 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:58:00.239391 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:58:00.239391 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 24 05:58:00.239391 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:58:00.239391 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:58:00.239391 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:58:00.239391 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:58:00.240440 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:58:00.240440 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:58:00.240440 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:58:00.240440 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:58:00.240440 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:58:00.240440 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:58:00.240440 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:58:00.240440 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:58:00.240440 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:58:00.240440 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:58:00.240440 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 05:58:00.240440 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 24 05:58:00.240440 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 362/729] 10.222.0.62 () {56 vars in 1102 bytes} [Wed Aug 24 05:57:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:58:00.252106 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d0d4e1-10df-48c7-8d9d-15182894a671 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:00.282363 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:00.283819 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:00.361224 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f55c19b-ebb5-4b24-8b01-95eb460b7a16 None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:00.395641 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d0d4e1-10df-48c7-8d9d-15182894a671 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:00.397553 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2bb0bd-ab59-4528-bf75-43b94b343458 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:00.399140 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2bb0bd-ab59-4528-bf75-43b94b343458 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:00.399649 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2bb0bd-ab59-4528-bf75-43b94b343458 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:00.400095 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2bb0bd-ab59-4528-bf75-43b94b343458 None admin] PATH_INFO: `/v3/users` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:00.401031 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2bb0bd-ab59-4528-bf75-43b94b343458 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:58:00.401509 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2bb0bd-ab59-4528-bf75-43b94b343458 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:00.403099 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2bb0bd-ab59-4528-bf75-43b94b343458 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:00.426041 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 368/730] 10.222.0.62 () {56 vars in 1135 bytes} [Wed Aug 24 05:58:00 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:58:00.427614 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341edcf3-b254-421f-bc81-e92a04034a6b None None] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:00.428057 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341edcf3-b254-421f-bc81-e92a04034a6b None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:00.428489 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341edcf3-b254-421f-bc81-e92a04034a6b None None] PATH_INFO: `/` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:00.430328 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 369/731] 10.222.0.62 () {54 vars in 927 bytes} [Wed Aug 24 05:58:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:58:00.431515 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341edcf3-b254-421f-bc81-e92a04034a6b None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:00.447730 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d74a8e-67d8-40d2-b014-28208966d80c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:00.448451 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d74a8e-67d8-40d2-b014-28208966d80c None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:00.448830 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d74a8e-67d8-40d2-b014-28208966d80c None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:00.449171 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d74a8e-67d8-40d2-b014-28208966d80c None admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:00.450158 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d74a8e-67d8-40d2-b014-28208966d80c None admin] RBAC: Adding query filter params (name=service) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:58:00.450530 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d74a8e-67d8-40d2-b014-28208966d80c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:00.453270 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d74a8e-67d8-40d2-b014-28208966d80c None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:00.468549 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 363/732] 10.222.0.62 () {56 vars in 1103 bytes} [Wed Aug 24 05:58:00 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:58:00.468549 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a65310-df50-43c1-9770-4f5d98723a79 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:00.468549 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a65310-df50-43c1-9770-4f5d98723a79 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:00.468549 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a65310-df50-43c1-9770-4f5d98723a79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:00.488414 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-71a65310-df50-43c1-9770-4f5d98723a79 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:58:00.490113 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-71a65310-df50-43c1-9770-4f5d98723a79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:58:00.549601 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:00.562594 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341edcf3-b254-421f-bc81-e92a04034a6b None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:00.581892 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 364/733] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Aug 24 05:58:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 05:58:00.581892 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a65310-df50-43c1-9770-4f5d98723a79 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:00.605409 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:00.629222 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5fac2a-f590-4d99-8128-6d791070e5c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:00.629837 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5fac2a-f590-4d99-8128-6d791070e5c0 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:00.630115 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5fac2a-f590-4d99-8128-6d791070e5c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:00.630386 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5fac2a-f590-4d99-8128-6d791070e5c0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:00.770872 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a65310-df50-43c1-9770-4f5d98723a79 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:00.807579 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db725a37-7646-45a8-bcd2-71772036bbe7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:00.808233 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db725a37-7646-45a8-bcd2-71772036bbe7 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:00.808597 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db725a37-7646-45a8-bcd2-71772036bbe7 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:00.808869 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db725a37-7646-45a8-bcd2-71772036bbe7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:00.820515 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db725a37-7646-45a8-bcd2-71772036bbe7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:00.822026 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db725a37-7646-45a8-bcd2-71772036bbe7 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:00.837957 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-db725a37-7646-45a8-bcd2-71772036bbe7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:58:00.837957 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:58:00.837957 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:58:00.837957 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:58:00.837957 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:58:00.837957 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:58:00.837957 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:58:00.837957 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:58:00.837957 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:58:00.837957 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:58:00.837957 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:58:00.837957 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:58:00.837957 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 05:58:00.837957 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 05:58:00.837957 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 05:58:00.837957 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 05:58:00.839156 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:58:00.839156 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:58:00.839156 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:58:00.839156 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:58:00.839156 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:58:00.839156 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:58:00.839156 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:58:00.839156 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 24 05:58:00.839156 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:58:00.839156 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:58:00.839156 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:58:00.839156 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:58:00.839156 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:58:00.839156 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:58:00.839156 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:58:00.839156 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:58:00.839156 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 05:58:00.840215 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 05:58:00.840215 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:58:00.840215 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 24 05:58:00.840215 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 365/734] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Aug 24 05:58:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:58:00.840705 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e483b5a0-a405-4011-9336-f1c4f84764e7 None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:00.840915 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e483b5a0-a405-4011-9336-f1c4f84764e7 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:00.841176 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e483b5a0-a405-4011-9336-f1c4f84764e7 None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:00.842510 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 366/735] 10.222.0.62 () {54 vars in 927 bytes} [Wed Aug 24 05:58:00 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:58:00.847579 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e483b5a0-a405-4011-9336-f1c4f84764e7 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:00.901942 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5fac2a-f590-4d99-8128-6d791070e5c0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:00.903521 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5fac2a-f590-4d99-8128-6d791070e5c0 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:00.923451 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:00.964540 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e483b5a0-a405-4011-9336-f1c4f84764e7 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:00.998442 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12025ef4-6e43-496d-927f-96b36ac9382f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:00.999618 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12025ef4-6e43-496d-927f-96b36ac9382f None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:00.999835 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12025ef4-6e43-496d-927f-96b36ac9382f None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:01.000103 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12025ef4-6e43-496d-927f-96b36ac9382f None admin] PATH_INFO: `/v3/domains` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:01.000934 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12025ef4-6e43-496d-927f-96b36ac9382f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:58:01.001234 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12025ef4-6e43-496d-927f-96b36ac9382f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:01.002712 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12025ef4-6e43-496d-927f-96b36ac9382f None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:01.006907 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:01.021278 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 367/736] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Aug 24 05:58:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:58:01.021278 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebea210-624c-4adc-b503-b1054b1a49a5 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:01.021278 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebea210-624c-4adc-b503-b1054b1a49a5 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:01.021278 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebea210-624c-4adc-b503-b1054b1a49a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:01.022050 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 370/737] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Aug 24 05:58:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 590 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 24 05:58:01.022731 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5fac2a-f590-4d99-8128-6d791070e5c0 None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:01.036690 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-6ebea210-624c-4adc-b503-b1054b1a49a5 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:58:01.037920 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-6ebea210-624c-4adc-b503-b1054b1a49a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:58:01.109373 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 368/738] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Aug 24 05:58:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 05:58:01.110204 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebea210-624c-4adc-b503-b1054b1a49a5 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:01.137633 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5fac2a-f590-4d99-8128-6d791070e5c0 None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:01.160373 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:01.178036 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db13043-abf9-4e42-901a-8697b06473fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:01.178727 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db13043-abf9-4e42-901a-8697b06473fa None admin] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:01.179011 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db13043-abf9-4e42-901a-8697b06473fa None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:01.179288 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db13043-abf9-4e42-901a-8697b06473fa None admin] PATH_INFO: `/v3/services` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:01.179943 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db13043-abf9-4e42-901a-8697b06473fa None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:01.181331 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db13043-abf9-4e42-901a-8697b06473fa None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:01.209941 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebea210-624c-4adc-b503-b1054b1a49a5 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:01.244236 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad52c10f-53ad-4904-9a17-c3a68cb2fce2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:01.244837 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad52c10f-53ad-4904-9a17-c3a68cb2fce2 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:01.245119 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad52c10f-53ad-4904-9a17-c3a68cb2fce2 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:01.245393 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad52c10f-53ad-4904-9a17-c3a68cb2fce2 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:01.252297 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad52c10f-53ad-4904-9a17-c3a68cb2fce2 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:01.253900 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad52c10f-53ad-4904-9a17-c3a68cb2fce2 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:01.265608 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-ad52c10f-53ad-4904-9a17-c3a68cb2fce2 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 05:58:01.265608 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:58:01.265608 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:58:01.265608 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:58:01.265608 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:58:01.265608 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:58:01.265608 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:58:01.265608 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:58:01.265608 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:58:01.265608 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:58:01.265608 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:58:01.265608 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:58:01.265608 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 05:58:01.265608 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 05:58:01.265608 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 05:58:01.265608 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 05:58:01.271279 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:58:01.271279 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:58:01.271279 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 05:58:01.271279 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 05:58:01.271279 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 05:58:01.271279 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 05:58:01.271279 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:58:01.271279 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:58:01.271279 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:58:01.271279 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:58:01.271279 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:58:01.271279 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 24 05:58:01.271279 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:58:01.271279 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:58:01.271279 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:58:01.271279 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:58:01.271279 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:58:01.271279 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:58:01.272452 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:58:01.272452 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:58:01.272452 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 05:58:01.272452 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 05:58:01.272452 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 05:58:01.272452 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 05:58:01.272452 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 05:58:01.272452 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 05:58:01.272452 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 05:58:01.272452 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 24 05:58:01.272452 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 369/739] 10.222.0.62 () {56 vars in 1117 bytes} [Wed Aug 24 05:58:01 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:58:01.272452 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad52c10f-53ad-4904-9a17-c3a68cb2fce2 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:01.281114 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 371/740] 10.222.0.62 () {60 vars in 1141 bytes} [Wed Aug 24 05:58:01 2022] POST /identity/v3/services => generated 251 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:58:01.282994 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07210f59-a6a5-42ee-b036-bd51ee9d5b8f None None] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:01.283467 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07210f59-a6a5-42ee-b036-bd51ee9d5b8f None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:01.284249 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07210f59-a6a5-42ee-b036-bd51ee9d5b8f None None] PATH_INFO: `/` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:01.286924 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 372/741] 10.222.0.62 () {54 vars in 927 bytes} [Wed Aug 24 05:58:01 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:58:01.288006 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07210f59-a6a5-42ee-b036-bd51ee9d5b8f None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:01.371606 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:01.400002 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad52c10f-53ad-4904-9a17-c3a68cb2fce2 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:01.416954 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:01.438687 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07210f59-a6a5-42ee-b036-bd51ee9d5b8f None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:01.440600 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d553ba4-7bc4-4b2d-84a9-e374723c974a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:01.441237 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d553ba4-7bc4-4b2d-84a9-e374723c974a None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:01.441513 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d553ba4-7bc4-4b2d-84a9-e374723c974a None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:01.441780 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d553ba4-7bc4-4b2d-84a9-e374723c974a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:01.453885 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d553ba4-7bc4-4b2d-84a9-e374723c974a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:01.455344 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d553ba4-7bc4-4b2d-84a9-e374723c974a None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:01.469827 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-5d553ba4-7bc4-4b2d-84a9-e374723c974a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:58:01.469827 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:58:01.469827 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:58:01.469827 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:58:01.469827 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:58:01.469827 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:58:01.469827 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:58:01.469827 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:58:01.469827 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:58:01.469827 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:58:01.469827 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:58:01.469827 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:58:01.469827 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 05:58:01.469827 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 05:58:01.469827 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 05:58:01.469827 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 05:58:01.470982 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:58:01.470982 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:58:01.470982 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:58:01.470982 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:58:01.470982 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:58:01.470982 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:58:01.470982 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:58:01.470982 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 24 05:58:01.470982 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:58:01.470982 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:58:01.470982 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:58:01.470982 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:58:01.470982 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:58:01.470982 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:58:01.470982 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:58:01.470982 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:58:01.470982 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 05:58:01.472084 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 05:58:01.472084 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:58:01.472084 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 24 05:58:01.473473 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 370/742] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Aug 24 05:58:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:58:01.474766 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a36fa0-7321-43a7-bb5c-043e87da5d46 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:01.475046 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a36fa0-7321-43a7-bb5c-043e87da5d46 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:01.475316 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a36fa0-7321-43a7-bb5c-043e87da5d46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:01.481087 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5f0ca1-7b8b-4461-9d39-f08f29661ce0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:01.481898 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5f0ca1-7b8b-4461-9d39-f08f29661ce0 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:01.482325 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5f0ca1-7b8b-4461-9d39-f08f29661ce0 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:01.482744 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5f0ca1-7b8b-4461-9d39-f08f29661ce0 None admin] PATH_INFO: `/v3/users` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:01.483724 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5f0ca1-7b8b-4461-9d39-f08f29661ce0 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:58:01.484206 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5f0ca1-7b8b-4461-9d39-f08f29661ce0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:01.485673 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5f0ca1-7b8b-4461-9d39-f08f29661ce0 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:01.494410 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-f3a36fa0-7321-43a7-bb5c-043e87da5d46 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:58:01.495517 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-f3a36fa0-7321-43a7-bb5c-043e87da5d46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:58:01.504363 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 373/743] 10.222.0.62 () {56 vars in 1149 bytes} [Wed Aug 24 05:58:01 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:58:01.505482 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5f0ca1-7b8b-4461-9d39-f08f29661ce0 None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:01.570036 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 371/744] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Aug 24 05:58:01 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 05:58:01.570999 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a36fa0-7321-43a7-bb5c-043e87da5d46 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:01.618404 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5f0ca1-7b8b-4461-9d39-f08f29661ce0 None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:01.621877 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:01.658762 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341db675-d6aa-4350-9db4-b124a899aa44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:01.659400 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341db675-d6aa-4350-9db4-b124a899aa44 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:01.659676 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341db675-d6aa-4350-9db4-b124a899aa44 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:01.659946 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341db675-d6aa-4350-9db4-b124a899aa44 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:01.670825 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341db675-d6aa-4350-9db4-b124a899aa44 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:58:01.676033 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application [None req-341db675-d6aa-4350-9db4-b124a899aa44 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 05:58:01.676033 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:58:01.676033 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:58:01.676033 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:58:01.676033 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:58:01.676033 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:58:01.676033 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:58:01.676033 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:58:01.676033 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:58:01.676033 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:58:01.676033 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:58:01.676033 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:58:01.676033 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:58:01.676033 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:58:01.676033 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:58:01.676033 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:58:01.676990 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:58:01.676990 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:58:01.676990 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:58:01.676990 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:58:01.676990 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:58:01.676990 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:58:01.676990 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:58:01.676990 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:58:01.676990 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:58:01.676990 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:58:01.676990 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:58:01.676990 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application with Lock( Aug 24 05:58:01.676990 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:58:01.676990 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:58:01.676990 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:58:01.676990 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:58:01.677824 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:58:01.677824 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:58:01.677824 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:58:01.677824 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:58:01.677824 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:58:01.677824 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:58:01.677824 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:58:01.677824 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:58:01.677824 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:58:01.677824 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:58:01.677824 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 05:58:01.677824 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Aug 24 05:58:01.681971 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 374/745] 10.222.0.62 () {56 vars in 1096 bytes} [Wed Aug 24 05:58:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:58:01.685458 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341db675-d6aa-4350-9db4-b124a899aa44 None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:01.701555 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:01.717033 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a36fa0-7321-43a7-bb5c-043e87da5d46 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:01.757709 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd13d98-a25d-4e70-8fdf-ec3793eb288c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:01.758581 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd13d98-a25d-4e70-8fdf-ec3793eb288c None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:01.758766 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd13d98-a25d-4e70-8fdf-ec3793eb288c None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:01.759139 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd13d98-a25d-4e70-8fdf-ec3793eb288c None admin] PATH_INFO: `/v3/domains` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:01.759885 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd13d98-a25d-4e70-8fdf-ec3793eb288c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:58:01.760290 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd13d98-a25d-4e70-8fdf-ec3793eb288c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:01.762195 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd13d98-a25d-4e70-8fdf-ec3793eb288c None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:01.775510 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 372/746] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Aug 24 05:58:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:58:01.776644 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd13d98-a25d-4e70-8fdf-ec3793eb288c None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:01.793734 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341db675-d6aa-4350-9db4-b124a899aa44 None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:01.829419 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c771ab-1918-44ad-9da8-f12578b3f8e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:01.830295 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c771ab-1918-44ad-9da8-f12578b3f8e8 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:01.830718 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c771ab-1918-44ad-9da8-f12578b3f8e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:01.831127 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c771ab-1918-44ad-9da8-f12578b3f8e8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:01.862229 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:01.890238 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd13d98-a25d-4e70-8fdf-ec3793eb288c None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:01.902353 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c771ab-1918-44ad-9da8-f12578b3f8e8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:01.903824 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c771ab-1918-44ad-9da8-f12578b3f8e8 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:01.927615 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b972e1-821f-41e7-8a3e-ce610f52e99d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:01.928221 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b972e1-821f-41e7-8a3e-ce610f52e99d None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:01.928635 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b972e1-821f-41e7-8a3e-ce610f52e99d None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:01.928910 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b972e1-821f-41e7-8a3e-ce610f52e99d None admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:01.929585 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b972e1-821f-41e7-8a3e-ce610f52e99d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:58:01.929887 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b972e1-821f-41e7-8a3e-ce610f52e99d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:01.931233 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b972e1-821f-41e7-8a3e-ce610f52e99d None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:01.941322 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 373/747] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Aug 24 05:58:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:58:01.942391 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b972e1-821f-41e7-8a3e-ce610f52e99d None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:01.961904 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:02.012786 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 375/748] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Aug 24 05:58:01 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 05:58:02.013850 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c771ab-1918-44ad-9da8-f12578b3f8e8 None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:02.051356 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b972e1-821f-41e7-8a3e-ce610f52e99d None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:02.075219 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:02.087189 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04222c7c-1ea0-4ffb-b8a5-e5513b8d7719 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:02.087811 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04222c7c-1ea0-4ffb-b8a5-e5513b8d7719 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:02.088088 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04222c7c-1ea0-4ffb-b8a5-e5513b8d7719 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:02.088359 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04222c7c-1ea0-4ffb-b8a5-e5513b8d7719 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:02.094914 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04222c7c-1ea0-4ffb-b8a5-e5513b8d7719 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:02.096483 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04222c7c-1ea0-4ffb-b8a5-e5513b8d7719 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:02.107416 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-04222c7c-1ea0-4ffb-b8a5-e5513b8d7719 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 05:58:02.107416 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:58:02.107416 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:58:02.107416 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:58:02.107416 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:58:02.107416 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:58:02.107416 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:58:02.107416 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:58:02.107416 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:58:02.107416 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:58:02.107416 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:58:02.107416 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:58:02.107416 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 05:58:02.107416 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 05:58:02.107416 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 05:58:02.108863 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 05:58:02.108863 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:58:02.108863 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:58:02.108863 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:58:02.108863 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:58:02.108863 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:58:02.108863 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:58:02.108863 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:58:02.108863 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 24 05:58:02.108863 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:58:02.108863 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:58:02.108863 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:58:02.108863 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:58:02.108863 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:58:02.108863 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:58:02.108863 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:58:02.108863 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:58:02.109748 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 05:58:02.109748 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 05:58:02.109748 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 05:58:02.109748 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 05:58:02.109748 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 05:58:02.109748 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 05:58:02.109748 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 05:58:02.109748 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 24 05:58:02.109748 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 374/749] 10.222.0.62 () {56 vars in 1111 bytes} [Wed Aug 24 05:58:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:58:02.110560 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04222c7c-1ea0-4ffb-b8a5-e5513b8d7719 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:02.119162 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c771ab-1918-44ad-9da8-f12578b3f8e8 None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:02.155706 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5d129c-6334-4b5f-b545-755c7ebf28e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:02.156688 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5d129c-6334-4b5f-b545-755c7ebf28e0 None admin] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:02.157125 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5d129c-6334-4b5f-b545-755c7ebf28e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:02.157413 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5d129c-6334-4b5f-b545-755c7ebf28e0 None admin] PATH_INFO: `/v3/services` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:02.158053 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5d129c-6334-4b5f-b545-755c7ebf28e0 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:02.159414 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5d129c-6334-4b5f-b545-755c7ebf28e0 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:02.189744 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:02.211183 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04222c7c-1ea0-4ffb-b8a5-e5513b8d7719 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:02.244399 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1bb775-eb8c-4dcf-a82a-1ea7417a7c98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:02.245067 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1bb775-eb8c-4dcf-a82a-1ea7417a7c98 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:02.245442 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1bb775-eb8c-4dcf-a82a-1ea7417a7c98 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:02.245699 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1bb775-eb8c-4dcf-a82a-1ea7417a7c98 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:02.255376 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1bb775-eb8c-4dcf-a82a-1ea7417a7c98 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:02.256828 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1bb775-eb8c-4dcf-a82a-1ea7417a7c98 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:02.270953 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-9d1bb775-eb8c-4dcf-a82a-1ea7417a7c98 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:58:02.270953 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:58:02.270953 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:58:02.270953 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:58:02.270953 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:58:02.270953 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:58:02.270953 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:58:02.270953 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:58:02.270953 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:58:02.270953 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:58:02.270953 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:58:02.270953 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:58:02.270953 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 05:58:02.270953 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 05:58:02.270953 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 05:58:02.270953 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 05:58:02.272041 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:58:02.272041 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:58:02.272041 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:58:02.272041 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:58:02.272041 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:58:02.272041 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:58:02.272041 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:58:02.272041 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 24 05:58:02.272041 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:58:02.272041 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:58:02.272041 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:58:02.272041 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:58:02.272041 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:58:02.272041 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:58:02.272041 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:58:02.272041 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:58:02.272041 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 05:58:02.272975 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 05:58:02.272975 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:58:02.272975 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 24 05:58:02.274774 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 375/750] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Aug 24 05:58:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:58:02.274774 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1bb775-eb8c-4dcf-a82a-1ea7417a7c98 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:02.283452 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 376/751] 10.222.0.62 () {60 vars in 1141 bytes} [Wed Aug 24 05:58:02 2022] POST /identity/v3/services => generated 240 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:58:02.284804 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5d129c-6334-4b5f-b545-755c7ebf28e0 None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:02.400339 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1bb775-eb8c-4dcf-a82a-1ea7417a7c98 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:02.442865 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5d129c-6334-4b5f-b545-755c7ebf28e0 None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:02.443806 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:02.454104 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80b86d9-1ef5-492b-b7f8-eedd93be6fa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:02.455315 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80b86d9-1ef5-492b-b7f8-eedd93be6fa8 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:02.455745 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80b86d9-1ef5-492b-b7f8-eedd93be6fa8 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:02.456465 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80b86d9-1ef5-492b-b7f8-eedd93be6fa8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:02.468223 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:02.468341 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80b86d9-1ef5-492b-b7f8-eedd93be6fa8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:02.470614 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80b86d9-1ef5-492b-b7f8-eedd93be6fa8 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:02.488175 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3812931-c066-40f0-bdb0-22d5347c97d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:02.489266 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3812931-c066-40f0-bdb0-22d5347c97d2 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:02.489266 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3812931-c066-40f0-bdb0-22d5347c97d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:02.489435 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3812931-c066-40f0-bdb0-22d5347c97d2 None admin] PATH_INFO: `/v3/projects` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:02.489535 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-d80b86d9-1ef5-492b-b7f8-eedd93be6fa8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:58:02.489535 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:58:02.489535 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:58:02.489535 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:58:02.489535 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:58:02.489535 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:58:02.489535 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:58:02.489535 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:58:02.489535 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:58:02.489535 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:58:02.489535 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:58:02.489535 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:58:02.489535 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 05:58:02.489535 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 05:58:02.489535 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 05:58:02.489535 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 05:58:02.490373 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:58:02.490373 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:58:02.490373 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:58:02.490373 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:58:02.490373 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:58:02.490373 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:58:02.490373 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:58:02.490373 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 24 05:58:02.490373 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:58:02.490373 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:58:02.490373 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:58:02.490373 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:58:02.490373 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:58:02.490373 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:58:02.490373 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:58:02.490373 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:58:02.490373 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 05:58:02.491263 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 05:58:02.491263 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:58:02.491263 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 24 05:58:02.491430 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3812931-c066-40f0-bdb0-22d5347c97d2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:58:02.491430 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3812931-c066-40f0-bdb0-22d5347c97d2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:02.493009 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3812931-c066-40f0-bdb0-22d5347c97d2 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:02.495288 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 376/752] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Aug 24 05:58:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:58:02.496849 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80b86d9-1ef5-492b-b7f8-eedd93be6fa8 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:02.508181 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 377/753] 10.222.0.62 () {56 vars in 1148 bytes} [Wed Aug 24 05:58:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:58:02.513516 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3812931-c066-40f0-bdb0-22d5347c97d2 None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:02.620384 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3812931-c066-40f0-bdb0-22d5347c97d2 None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:02.630663 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80b86d9-1ef5-492b-b7f8-eedd93be6fa8 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:02.657158 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819100cf-7e4e-4fbc-9ad4-6f26af52ad2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:02.658058 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819100cf-7e4e-4fbc-9ad4-6f26af52ad2b None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:02.658536 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819100cf-7e4e-4fbc-9ad4-6f26af52ad2b None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:02.659017 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819100cf-7e4e-4fbc-9ad4-6f26af52ad2b None admin] PATH_INFO: `/v3/domains` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:02.659855 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819100cf-7e4e-4fbc-9ad4-6f26af52ad2b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:58:02.660351 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819100cf-7e4e-4fbc-9ad4-6f26af52ad2b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:02.661983 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819100cf-7e4e-4fbc-9ad4-6f26af52ad2b None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:02.666518 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc66535-d32f-49a2-92f6-d018bda9c946 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:02.667135 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc66535-d32f-49a2-92f6-d018bda9c946 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:02.667415 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc66535-d32f-49a2-92f6-d018bda9c946 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:02.667684 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc66535-d32f-49a2-92f6-d018bda9c946 None admin] PATH_INFO: `/v3/domains` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:02.668319 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc66535-d32f-49a2-92f6-d018bda9c946 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:58:02.668654 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc66535-d32f-49a2-92f6-d018bda9c946 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:02.670284 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc66535-d32f-49a2-92f6-d018bda9c946 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:02.677272 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 378/754] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Aug 24 05:58:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:58:02.678401 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819100cf-7e4e-4fbc-9ad4-6f26af52ad2b None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:02.692482 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 377/755] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Aug 24 05:58:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:58:02.693710 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc66535-d32f-49a2-92f6-d018bda9c946 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:02.727978 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:02.754079 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:02.795983 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819100cf-7e4e-4fbc-9ad4-6f26af52ad2b None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:02.835690 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc66535-d32f-49a2-92f6-d018bda9c946 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:02.840166 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e618cc59-9b08-44be-b503-cc65e019ad2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:02.840861 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e618cc59-9b08-44be-b503-cc65e019ad2f None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:02.841142 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e618cc59-9b08-44be-b503-cc65e019ad2f None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:02.841464 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e618cc59-9b08-44be-b503-cc65e019ad2f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:02.842191 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e618cc59-9b08-44be-b503-cc65e019ad2f None admin] RBAC: Adding query filter params (role.id=49e47cd157d743f6905f1db07ec878a7, scope.project.id=181f4762037e454489d9552cc603179c, user.id=14d4fed04fb144d08918bd5d4f808d23) {{(pid=81851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:58:02.842498 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e618cc59-9b08-44be-b503-cc65e019ad2f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:02.844525 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e618cc59-9b08-44be-b503-cc65e019ad2f None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:02.860530 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 379/756] 10.222.0.62 () {56 vars in 1374 bytes} [Wed Aug 24 05:58:02 2022] GET /identity/v3/role_assignments?user.id=14d4fed04fb144d08918bd5d4f808d23&scope.project.id=181f4762037e454489d9552cc603179c&role.id=49e47cd157d743f6905f1db07ec878a7 => generated 259 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:58:02.860530 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e618cc59-9b08-44be-b503-cc65e019ad2f None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:02.929323 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e573ab5-98d2-4284-90e0-657cce3b03be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:02.930729 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e573ab5-98d2-4284-90e0-657cce3b03be None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:02.931238 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e573ab5-98d2-4284-90e0-657cce3b03be None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:02.931817 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e573ab5-98d2-4284-90e0-657cce3b03be None admin] PATH_INFO: `/v3/projects/service` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:02.945699 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e573ab5-98d2-4284-90e0-657cce3b03be None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:02.947517 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e573ab5-98d2-4284-90e0-657cce3b03be None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:02.970494 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-9e573ab5-98d2-4284-90e0-657cce3b03be None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 05:58:02.970494 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:58:02.970494 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:58:02.970494 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:58:02.970494 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:58:02.970494 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:58:02.970494 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:58:02.970494 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:58:02.970494 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:58:02.970494 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:58:02.970494 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:58:02.970494 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:58:02.970494 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 05:58:02.970494 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 05:58:02.970494 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 05:58:02.971536 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 05:58:02.971536 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:58:02.971536 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:58:02.971536 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:58:02.971536 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:58:02.971536 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:58:02.971536 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:58:02.971536 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:58:02.971536 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 24 05:58:02.971536 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:58:02.971536 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:58:02.971536 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:58:02.971536 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:58:02.971536 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:58:02.971536 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:58:02.971536 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:58:02.971536 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:58:02.972429 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 05:58:02.972429 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 05:58:02.972429 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 05:58:02.972429 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 05:58:02.972429 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 05:58:02.972429 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 05:58:02.972429 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 05:58:02.972429 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 24 05:58:02.981018 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e618cc59-9b08-44be-b503-cc65e019ad2f None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:02.981018 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:02.981999 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 378/757] 10.222.0.62 () {56 vars in 1111 bytes} [Wed Aug 24 05:58:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:58:02.995040 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e573ab5-98d2-4284-90e0-657cce3b03be None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:03.025267 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1947b2-d363-4c09-86df-b8f357dfd90e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:03.027582 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1947b2-d363-4c09-86df-b8f357dfd90e None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:03.027979 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1947b2-d363-4c09-86df-b8f357dfd90e None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:03.028329 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1947b2-d363-4c09-86df-b8f357dfd90e None admin] PATH_INFO: `/v3/users/nova` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:03.037658 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1947b2-d363-4c09-86df-b8f357dfd90e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:03.039233 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1947b2-d363-4c09-86df-b8f357dfd90e None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:03.053839 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application [None req-5d1947b2-d363-4c09-86df-b8f357dfd90e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 05:58:03.053839 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:58:03.053839 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:58:03.053839 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:58:03.053839 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:58:03.053839 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:58:03.053839 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:58:03.053839 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:58:03.053839 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:58:03.053839 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:58:03.053839 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:58:03.053839 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:58:03.053839 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 05:58:03.053839 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 05:58:03.053839 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 05:58:03.053839 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 05:58:03.054838 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:58:03.054838 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:58:03.054838 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 05:58:03.054838 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 05:58:03.054838 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 05:58:03.054838 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 05:58:03.054838 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:58:03.054838 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:58:03.054838 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:58:03.054838 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:58:03.054838 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:58:03.054838 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application with Lock( Aug 24 05:58:03.054838 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:58:03.054838 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:58:03.054838 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:58:03.054838 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:58:03.054838 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:58:03.054838 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:58:03.055790 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:58:03.055790 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:58:03.055790 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 05:58:03.055790 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 05:58:03.055790 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 05:58:03.055790 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 05:58:03.055790 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 05:58:03.055790 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 05:58:03.055790 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 05:58:03.055790 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Aug 24 05:58:03.055790 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 380/758] 10.222.0.62 () {56 vars in 1093 bytes} [Wed Aug 24 05:58:02 2022] GET /identity/v3/users/nova => generated 82 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:58:03.060308 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1947b2-d363-4c09-86df-b8f357dfd90e None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:03.187310 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1947b2-d363-4c09-86df-b8f357dfd90e None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:03.205942 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:03.214412 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e573ab5-98d2-4284-90e0-657cce3b03be None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:03.228907 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed90b32-506e-4e89-9a1e-b48631719863 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:03.229883 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed90b32-506e-4e89-9a1e-b48631719863 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:03.230377 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed90b32-506e-4e89-9a1e-b48631719863 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:03.231226 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed90b32-506e-4e89-9a1e-b48631719863 None admin] PATH_INFO: `/v3/users` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:03.232293 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed90b32-506e-4e89-9a1e-b48631719863 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:58:03.232829 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed90b32-506e-4e89-9a1e-b48631719863 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:03.234575 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed90b32-506e-4e89-9a1e-b48631719863 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:03.256350 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 381/759] 10.222.0.62 () {56 vars in 1133 bytes} [Wed Aug 24 05:58:03 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:58:03.262186 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed90b32-506e-4e89-9a1e-b48631719863 None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:03.298401 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05eb2039-a50c-4305-af77-35fffeeb24f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:03.299429 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05eb2039-a50c-4305-af77-35fffeeb24f5 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:03.300369 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05eb2039-a50c-4305-af77-35fffeeb24f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:03.300867 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05eb2039-a50c-4305-af77-35fffeeb24f5 None admin] PATH_INFO: `/v3/projects` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:03.302001 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05eb2039-a50c-4305-af77-35fffeeb24f5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:58:03.302504 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05eb2039-a50c-4305-af77-35fffeeb24f5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:03.302626 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:03.305400 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05eb2039-a50c-4305-af77-35fffeeb24f5 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:03.326811 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 379/760] 10.222.0.62 () {56 vars in 1148 bytes} [Wed Aug 24 05:58:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:58:03.329074 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2973f7-6db7-4a45-914f-c1defc6847ab None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:03.329588 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2973f7-6db7-4a45-914f-c1defc6847ab None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:03.330060 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2973f7-6db7-4a45-914f-c1defc6847ab None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:03.332742 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 380/761] 10.222.0.62 () {54 vars in 927 bytes} [Wed Aug 24 05:58:03 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:58:03.333771 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2973f7-6db7-4a45-914f-c1defc6847ab None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:03.413907 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed90b32-506e-4e89-9a1e-b48631719863 None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:03.467481 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e0ba75-5822-45d0-8c9f-d9b4bd8e5067 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:03.468061 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e0ba75-5822-45d0-8c9f-d9b4bd8e5067 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:03.468338 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e0ba75-5822-45d0-8c9f-d9b4bd8e5067 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:03.468632 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e0ba75-5822-45d0-8c9f-d9b4bd8e5067 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:03.474016 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2973f7-6db7-4a45-914f-c1defc6847ab None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:03.479480 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e0ba75-5822-45d0-8c9f-d9b4bd8e5067 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:03.480957 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e0ba75-5822-45d0-8c9f-d9b4bd8e5067 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:03.496391 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application [None req-b5e0ba75-5822-45d0-8c9f-d9b4bd8e5067 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:58:03.496391 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:58:03.496391 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:58:03.496391 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:58:03.496391 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:58:03.496391 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:58:03.496391 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:58:03.496391 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:58:03.496391 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:58:03.496391 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:58:03.496391 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:58:03.496391 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:58:03.496391 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 05:58:03.496391 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 05:58:03.496391 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 05:58:03.496391 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 05:58:03.497372 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:58:03.497372 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:58:03.497372 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:58:03.497372 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:58:03.497372 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:58:03.497372 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:58:03.497372 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:58:03.497372 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application with Lock( Aug 24 05:58:03.497372 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:58:03.497372 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:58:03.497372 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:58:03.497372 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:58:03.497372 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:58:03.497372 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:58:03.497372 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:58:03.497372 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:58:03.497372 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 05:58:03.499464 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 05:58:03.499464 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:58:03.499464 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Aug 24 05:58:03.499464 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 382/762] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Aug 24 05:58:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:58:03.499711 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca0bd9b-2fc0-46a7-bb92-c8f56b5ddf6a None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:03.499951 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca0bd9b-2fc0-46a7-bb92-c8f56b5ddf6a None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:03.500139 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca0bd9b-2fc0-46a7-bb92-c8f56b5ddf6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:03.516818 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9a3567-bf8d-45ea-a521-0120db22af07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:03.517574 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9a3567-bf8d-45ea-a521-0120db22af07 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:03.517918 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9a3567-bf8d-45ea-a521-0120db22af07 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:03.518098 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-8ca0bd9b-2fc0-46a7-bb92-c8f56b5ddf6a None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:58:03.518280 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9a3567-bf8d-45ea-a521-0120db22af07 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:03.518961 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9a3567-bf8d-45ea-a521-0120db22af07 None admin] RBAC: Adding query filter params (role.id=5d95a5fc0f324a92905533aa8d34ac15, scope.project.id=181f4762037e454489d9552cc603179c, user.id=b82726a12436435cb931b986ef884c2e) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:58:03.519136 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-8ca0bd9b-2fc0-46a7-bb92-c8f56b5ddf6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:58:03.519300 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9a3567-bf8d-45ea-a521-0120db22af07 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:03.520816 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9a3567-bf8d-45ea-a521-0120db22af07 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:03.529201 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 381/763] 10.222.0.62 () {56 vars in 1374 bytes} [Wed Aug 24 05:58:03 2022] GET /identity/v3/role_assignments?user.id=b82726a12436435cb931b986ef884c2e&scope.project.id=181f4762037e454489d9552cc603179c&role.id=5d95a5fc0f324a92905533aa8d34ac15 => generated 259 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:58:03.530222 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9a3567-bf8d-45ea-a521-0120db22af07 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:03.569561 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:03.593787 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 383/764] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Aug 24 05:58:03 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 05:58:03.660457 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9a3567-bf8d-45ea-a521-0120db22af07 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:03.699218 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6e618d-f726-470b-bfeb-190e2c4616d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:03.700150 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6e618d-f726-470b-bfeb-190e2c4616d1 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:03.700667 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6e618d-f726-470b-bfeb-190e2c4616d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:03.701114 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6e618d-f726-470b-bfeb-190e2c4616d1 None admin] PATH_INFO: `/v3/domains` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:03.702051 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6e618d-f726-470b-bfeb-190e2c4616d1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:58:03.702507 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6e618d-f726-470b-bfeb-190e2c4616d1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:03.704057 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6e618d-f726-470b-bfeb-190e2c4616d1 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:03.717401 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 382/765] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Aug 24 05:58:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:58:03.723686 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca0bd9b-2fc0-46a7-bb92-c8f56b5ddf6a None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:03.735499 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:03.856132 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca0bd9b-2fc0-46a7-bb92-c8f56b5ddf6a None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:03.891418 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f45459-b504-44d3-b620-b7127ac0fd0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:03.892334 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f45459-b504-44d3-b620-b7127ac0fd0d None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:03.892835 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f45459-b504-44d3-b620-b7127ac0fd0d None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:03.893280 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f45459-b504-44d3-b620-b7127ac0fd0d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:03.900277 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f45459-b504-44d3-b620-b7127ac0fd0d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:03.902020 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f45459-b504-44d3-b620-b7127ac0fd0d None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:03.913724 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application [None req-17f45459-b504-44d3-b620-b7127ac0fd0d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 05:58:03.913724 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:58:03.913724 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:58:03.913724 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:58:03.913724 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:58:03.913724 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:58:03.913724 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:58:03.913724 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:58:03.913724 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:58:03.913724 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:58:03.913724 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:58:03.913724 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:58:03.913724 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 05:58:03.913724 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 05:58:03.913724 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 05:58:03.914614 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 05:58:03.914614 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:58:03.914614 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:58:03.914614 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:58:03.914614 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:58:03.914614 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:58:03.914614 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:58:03.914614 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:58:03.914614 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application with Lock( Aug 24 05:58:03.914614 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:58:03.914614 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:58:03.914614 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:58:03.914614 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:58:03.914614 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:58:03.914614 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:58:03.914614 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:58:03.914614 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:58:03.915486 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 05:58:03.915486 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 05:58:03.915486 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 05:58:03.915486 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 05:58:03.915486 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 05:58:03.915486 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 05:58:03.915486 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 05:58:03.915486 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Aug 24 05:58:03.917408 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 384/766] 10.222.0.62 () {56 vars in 1111 bytes} [Wed Aug 24 05:58:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:58:03.924059 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6e618d-f726-470b-bfeb-190e2c4616d1 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:04.002469 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f45459-b504-44d3-b620-b7127ac0fd0d None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:04.056182 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:04.072019 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6e618d-f726-470b-bfeb-190e2c4616d1 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:04.107929 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2997d335-d901-4674-9323-dc87fe2d9ad7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:04.109131 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2997d335-d901-4674-9323-dc87fe2d9ad7 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:04.109413 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2997d335-d901-4674-9323-dc87fe2d9ad7 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:04.109681 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2997d335-d901-4674-9323-dc87fe2d9ad7 None admin] PATH_INFO: `/v3/projects` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:04.110501 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2997d335-d901-4674-9323-dc87fe2d9ad7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:58:04.110860 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2997d335-d901-4674-9323-dc87fe2d9ad7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:04.112409 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2997d335-d901-4674-9323-dc87fe2d9ad7 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:04.127495 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 383/767] 10.222.0.62 () {56 vars in 1148 bytes} [Wed Aug 24 05:58:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:58:04.133287 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:04.134224 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2997d335-d901-4674-9323-dc87fe2d9ad7 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:04.251555 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2997d335-d901-4674-9323-dc87fe2d9ad7 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:04.278535 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f45459-b504-44d3-b620-b7127ac0fd0d None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:04.289680 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d7e179-6d45-497f-b437-8da00bab7bc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:04.290356 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d7e179-6d45-497f-b437-8da00bab7bc3 None admin] REQUEST_METHOD: `PUT` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:04.290642 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d7e179-6d45-497f-b437-8da00bab7bc3 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:04.290974 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d7e179-6d45-497f-b437-8da00bab7bc3 None admin] PATH_INFO: `/v3/projects/181f4762037e454489d9552cc603179c/users/fb6d3743340241f98cacb9e0d063298c/roles/49e47cd157d743f6905f1db07ec878a7` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:04.292919 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d7e179-6d45-497f-b437-8da00bab7bc3 None admin] RBAC: Authorizing `identity:create_grant(project_id=181f4762037e454489d9552cc603179c, user_id=fb6d3743340241f98cacb9e0d063298c, role_id=49e47cd157d743f6905f1db07ec878a7)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:04.294353 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d7e179-6d45-497f-b437-8da00bab7bc3 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:04.332753 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8248e1-f0c8-4d36-a86b-91abea3ff4c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:04.333424 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8248e1-f0c8-4d36-a86b-91abea3ff4c7 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:04.333648 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8248e1-f0c8-4d36-a86b-91abea3ff4c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:04.333917 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8248e1-f0c8-4d36-a86b-91abea3ff4c7 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:04.334694 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8248e1-f0c8-4d36-a86b-91abea3ff4c7 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:04.336089 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8248e1-f0c8-4d36-a86b-91abea3ff4c7 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:04.346436 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application [None req-fa8248e1-f0c8-4d36-a86b-91abea3ff4c7 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 05:58:04.346436 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:58:04.346436 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 05:58:04.346436 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 05:58:04.346436 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 05:58:04.346436 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 05:58:04.346436 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 05:58:04.346436 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 05:58:04.346436 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 05:58:04.346436 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Aug 24 05:58:04.346436 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 05:58:04.346436 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Aug 24 05:58:04.346436 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:58:04.346436 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:58:04.346436 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:58:04.346436 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:58:04.346436 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:58:04.353265 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:58:04.353265 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:58:04.353265 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:58:04.353265 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:58:04.353265 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:58:04.353265 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:58:04.353265 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 05:58:04.353265 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 05:58:04.353265 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 05:58:04.353265 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 05:58:04.353265 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:58:04.353265 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:58:04.353265 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:58:04.353265 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:58:04.353265 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:58:04.353265 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:58:04.354251 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:58:04.354251 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application with Lock( Aug 24 05:58:04.354251 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:58:04.354251 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:58:04.354251 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:58:04.354251 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:58:04.354251 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:58:04.354251 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:58:04.354251 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:58:04.354251 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:58:04.354251 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 05:58:04.354251 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 05:58:04.354251 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 05:58:04.354251 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Aug 24 05:58:04.354251 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 385/768] 10.222.0.62 () {56 vars in 1129 bytes} [Wed Aug 24 05:58:04 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 346 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 05:58:04.355217 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8248e1-f0c8-4d36-a86b-91abea3ff4c7 None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:04.380489 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 384/769] 10.222.0.62 () {58 vars in 1439 bytes} [Wed Aug 24 05:58:04 2022] PUT /identity/v3/projects/181f4762037e454489d9552cc603179c/users/fb6d3743340241f98cacb9e0d063298c/roles/49e47cd157d743f6905f1db07ec878a7 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 05:58:04.469287 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8248e1-f0c8-4d36-a86b-91abea3ff4c7 None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:04.497583 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:04.505979 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92e31fd-6b1e-42df-88b1-14723cf6a979 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:04.506598 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92e31fd-6b1e-42df-88b1-14723cf6a979 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:04.506872 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92e31fd-6b1e-42df-88b1-14723cf6a979 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:04.507240 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92e31fd-6b1e-42df-88b1-14723cf6a979 None admin] PATH_INFO: `/v3/services` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:04.512867 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92e31fd-6b1e-42df-88b1-14723cf6a979 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:58:04.512867 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92e31fd-6b1e-42df-88b1-14723cf6a979 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:04.512867 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92e31fd-6b1e-42df-88b1-14723cf6a979 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:04.515826 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 386/770] 10.222.0.62 () {56 vars in 1124 bytes} [Wed Aug 24 05:58:04 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:58:04.521345 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d7e179-6d45-497f-b437-8da00bab7bc3 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:04.533903 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:04.557273 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12372658-1045-4e74-8671-707129f87410 None None] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:04.557730 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12372658-1045-4e74-8671-707129f87410 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:04.558020 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12372658-1045-4e74-8671-707129f87410 None None] PATH_INFO: `/` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:04.559762 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 387/771] 10.222.0.62 () {54 vars in 927 bytes} [Wed Aug 24 05:58:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:58:04.566870 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd753db-e504-4b57-bce7-2064b00a7e8c None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:04.567152 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd753db-e504-4b57-bce7-2064b00a7e8c None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:04.567423 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd753db-e504-4b57-bce7-2064b00a7e8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:04.589842 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-bcd753db-e504-4b57-bce7-2064b00a7e8c None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:58:04.591170 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-bcd753db-e504-4b57-bce7-2064b00a7e8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:58:04.679378 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 388/772] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Aug 24 05:58:04 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 05:58:04.721852 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d7e179-6d45-497f-b437-8da00bab7bc3 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:04.757029 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366c1003-498a-4dae-b95d-b1707cd9b4df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:04.758022 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366c1003-498a-4dae-b95d-b1707cd9b4df None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:04.758502 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366c1003-498a-4dae-b95d-b1707cd9b4df None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:04.758991 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366c1003-498a-4dae-b95d-b1707cd9b4df None admin] PATH_INFO: `/v3/services` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:04.759850 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366c1003-498a-4dae-b95d-b1707cd9b4df None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:58:04.760362 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366c1003-498a-4dae-b95d-b1707cd9b4df None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:04.762152 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366c1003-498a-4dae-b95d-b1707cd9b4df None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:04.770372 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 385/773] 10.222.0.62 () {56 vars in 1124 bytes} [Wed Aug 24 05:58:04 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:58:04.776580 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd753db-e504-4b57-bce7-2064b00a7e8c None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:04.921121 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd753db-e504-4b57-bce7-2064b00a7e8c None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:04.945101 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:04.957364 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bc432a-f9eb-49d1-b7c2-7994ea819346 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:04.957980 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bc432a-f9eb-49d1-b7c2-7994ea819346 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:04.958254 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bc432a-f9eb-49d1-b7c2-7994ea819346 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:04.964710 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bc432a-f9eb-49d1-b7c2-7994ea819346 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:04.964710 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82bc432a-f9eb-49d1-b7c2-7994ea819346 None admin] RBAC: Adding query filter params (interface=public, service_id=a0210735ceb04248a7bb47e8abc2a796, region_id=RegionOne) {{(pid=81851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:58:04.964710 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82bc432a-f9eb-49d1-b7c2-7994ea819346 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:04.964710 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82bc432a-f9eb-49d1-b7c2-7994ea819346 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:04.969166 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 389/774] 10.222.0.62 () {56 vars in 1251 bytes} [Wed Aug 24 05:58:04 2022] GET /identity/v3/endpoints?service_id=a0210735ceb04248a7bb47e8abc2a796&interface=public®ion_id=RegionOne => generated 194 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:58:04.974262 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366c1003-498a-4dae-b95d-b1707cd9b4df None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:05.081844 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:05.096096 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bc432a-f9eb-49d1-b7c2-7994ea819346 None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:05.126412 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366c1003-498a-4dae-b95d-b1707cd9b4df None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:05.192074 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7b5b1b-52a9-418a-bf76-4839b6ad6c10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:05.193075 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7b5b1b-52a9-418a-bf76-4839b6ad6c10 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:05.193532 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7b5b1b-52a9-418a-bf76-4839b6ad6c10 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:05.193974 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7b5b1b-52a9-418a-bf76-4839b6ad6c10 None admin] PATH_INFO: `/v3/services` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:05.194857 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7b5b1b-52a9-418a-bf76-4839b6ad6c10 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:05.196529 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7b5b1b-52a9-418a-bf76-4839b6ad6c10 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:05.208271 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 386/775] 10.222.0.62 () {56 vars in 1087 bytes} [Wed Aug 24 05:58:04 2022] GET /identity/v3/services => generated 772 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:58:05.212285 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bc432a-f9eb-49d1-b7c2-7994ea819346 None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:05.258519 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac31986-4841-4710-8411-1c52eef3d36b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:05.259706 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac31986-4841-4710-8411-1c52eef3d36b None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:05.260173 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac31986-4841-4710-8411-1c52eef3d36b None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:05.260671 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac31986-4841-4710-8411-1c52eef3d36b None admin] PATH_INFO: `/v3/services/network` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:05.261264 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac31986-4841-4710-8411-1c52eef3d36b None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:05.262637 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac31986-4841-4710-8411-1c52eef3d36b None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:05.271734 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application [None req-9ac31986-4841-4710-8411-1c52eef3d36b None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 05:58:05.271734 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:58:05.271734 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 05:58:05.271734 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 05:58:05.271734 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 05:58:05.271734 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 05:58:05.271734 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 05:58:05.271734 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 05:58:05.271734 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 05:58:05.271734 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Aug 24 05:58:05.271734 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 05:58:05.271734 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Aug 24 05:58:05.271734 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:58:05.271734 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:58:05.271734 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:58:05.271734 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:58:05.271734 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:58:05.272760 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:58:05.272760 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:58:05.272760 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:58:05.272760 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:58:05.272760 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:58:05.272760 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:58:05.272760 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 05:58:05.272760 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 05:58:05.272760 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 05:58:05.272760 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 05:58:05.272760 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:58:05.272760 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:58:05.272760 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:58:05.272760 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:58:05.272760 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:58:05.272760 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:58:05.273577 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:58:05.273577 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application with Lock( Aug 24 05:58:05.273577 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:58:05.273577 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:58:05.273577 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:58:05.273577 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:58:05.273577 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:58:05.273577 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:58:05.273577 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:58:05.273577 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:58:05.273577 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 05:58:05.273577 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 05:58:05.273577 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 05:58:05.273577 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Aug 24 05:58:05.273577 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 390/776] 10.222.0.62 () {56 vars in 1111 bytes} [Wed Aug 24 05:58:05 2022] GET /identity/v3/services/network => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 05:58:05.281853 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7b5b1b-52a9-418a-bf76-4839b6ad6c10 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:05.292085 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddf4b5a-5539-4b9f-aafe-e130035c84ff None None] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:05.292391 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddf4b5a-5539-4b9f-aafe-e130035c84ff None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:05.292673 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddf4b5a-5539-4b9f-aafe-e130035c84ff None None] PATH_INFO: `/` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:05.294016 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 391/777] 10.222.0.62 () {54 vars in 927 bytes} [Wed Aug 24 05:58:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 05:58:05.300119 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f99a6ae-a7a9-49e4-8b55-5bd4121275c5 None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:05.300413 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f99a6ae-a7a9-49e4-8b55-5bd4121275c5 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:05.300759 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f99a6ae-a7a9-49e4-8b55-5bd4121275c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:05.335899 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-5f99a6ae-a7a9-49e4-8b55-5bd4121275c5 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:58:05.339454 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-5f99a6ae-a7a9-49e4-8b55-5bd4121275c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:58:05.434105 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:05.457115 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7b5b1b-52a9-418a-bf76-4839b6ad6c10 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:05.464113 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 392/778] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Aug 24 05:58:05 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 05:58:05.496629 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:05.507126 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d337f5-aed2-4497-be8f-aa8e0bfd73d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:05.507988 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d337f5-aed2-4497-be8f-aa8e0bfd73d8 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:05.508427 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d337f5-aed2-4497-be8f-aa8e0bfd73d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:05.508904 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d337f5-aed2-4497-be8f-aa8e0bfd73d8 None admin] PATH_INFO: `/v3/services` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:05.509684 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d337f5-aed2-4497-be8f-aa8e0bfd73d8 None admin] RBAC: Adding query filter params (name=network) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:58:05.510924 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d337f5-aed2-4497-be8f-aa8e0bfd73d8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:05.511799 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d337f5-aed2-4497-be8f-aa8e0bfd73d8 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:05.517211 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 387/779] 10.222.0.62 () {56 vars in 1112 bytes} [Wed Aug 24 05:58:05 2022] GET /identity/v3/services?name=network => generated 124 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:58:05.523323 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f99a6ae-a7a9-49e4-8b55-5bd4121275c5 None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:05.643450 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f99a6ae-a7a9-49e4-8b55-5bd4121275c5 None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:05.681411 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52138ac5-df60-4eec-9d0c-bf4f330f819f None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:05.681745 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52138ac5-df60-4eec-9d0c-bf4f330f819f None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:05.681957 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52138ac5-df60-4eec-9d0c-bf4f330f819f None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:05.685450 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb29c56-4f4f-4295-8e2b-43cd2fec205a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:05.685582 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 388/780] 10.222.0.62 () {54 vars in 927 bytes} [Wed Aug 24 05:58:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 05:58:05.685750 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb29c56-4f4f-4295-8e2b-43cd2fec205a None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:05.685750 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb29c56-4f4f-4295-8e2b-43cd2fec205a None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:05.685750 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb29c56-4f4f-4295-8e2b-43cd2fec205a None admin] PATH_INFO: `/v3/services` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:05.685750 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb29c56-4f4f-4295-8e2b-43cd2fec205a None admin] RBAC: Adding query filter params (type=network) {{(pid=81851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:58:05.685994 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb29c56-4f4f-4295-8e2b-43cd2fec205a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:05.687425 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb29c56-4f4f-4295-8e2b-43cd2fec205a None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:05.692543 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987e8be4-efb6-4639-acdc-96c35ed57453 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:05.692543 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987e8be4-efb6-4639-acdc-96c35ed57453 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:05.692543 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987e8be4-efb6-4639-acdc-96c35ed57453 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:05.692798 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 393/781] 10.222.0.62 () {56 vars in 1112 bytes} [Wed Aug 24 05:58:05 2022] GET /identity/v3/services?type=network => generated 350 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:58:05.699582 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb29c56-4f4f-4295-8e2b-43cd2fec205a None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:05.719250 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-987e8be4-efb6-4639-acdc-96c35ed57453 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:58:05.720664 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-987e8be4-efb6-4639-acdc-96c35ed57453 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:58:05.790630 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:05.828529 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 389/782] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Aug 24 05:58:05 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 05:58:05.828529 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987e8be4-efb6-4639-acdc-96c35ed57453 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:05.841407 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb29c56-4f4f-4295-8e2b-43cd2fec205a None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:05.902644 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360ac208-3c5a-4ec0-b4c9-d76a1bd4c9ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:05.903673 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360ac208-3c5a-4ec0-b4c9-d76a1bd4c9ef None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:05.904148 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360ac208-3c5a-4ec0-b4c9-d76a1bd4c9ef None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:05.904706 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360ac208-3c5a-4ec0-b4c9-d76a1bd4c9ef None admin] PATH_INFO: `/v3/endpoints` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:05.906223 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360ac208-3c5a-4ec0-b4c9-d76a1bd4c9ef None admin] RBAC: Adding query filter params (interface=public, service_id=bb55156338f6452faee4966280181868, region_id=RegionOne) {{(pid=81851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:58:05.906837 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360ac208-3c5a-4ec0-b4c9-d76a1bd4c9ef None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:05.908510 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360ac208-3c5a-4ec0-b4c9-d76a1bd4c9ef None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:05.921628 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 394/783] 10.222.0.62 () {56 vars in 1251 bytes} [Wed Aug 24 05:58:05 2022] GET /identity/v3/endpoints?service_id=bb55156338f6452faee4966280181868&interface=public®ion_id=RegionOne => generated 194 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:58:05.927589 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360ac208-3c5a-4ec0-b4c9-d76a1bd4c9ef None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:05.947999 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987e8be4-efb6-4639-acdc-96c35ed57453 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:06.095668 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360ac208-3c5a-4ec0-b4c9-d76a1bd4c9ef None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:06.112579 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:06.149336 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:06.151907 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4bbbce-13cb-4b6d-88e6-555c64115290 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:06.152537 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4bbbce-13cb-4b6d-88e6-555c64115290 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:06.152826 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4bbbce-13cb-4b6d-88e6-555c64115290 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:06.153100 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4bbbce-13cb-4b6d-88e6-555c64115290 None admin] PATH_INFO: `/v3/services` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:06.153705 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4bbbce-13cb-4b6d-88e6-555c64115290 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:06.161843 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4bbbce-13cb-4b6d-88e6-555c64115290 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:06.164298 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f117768-a0ce-413e-aa1a-802be65fe061 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:06.165198 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f117768-a0ce-413e-aa1a-802be65fe061 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:06.165700 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f117768-a0ce-413e-aa1a-802be65fe061 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:06.166167 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f117768-a0ce-413e-aa1a-802be65fe061 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:06.179657 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 395/784] 10.222.0.62 () {56 vars in 1087 bytes} [Wed Aug 24 05:58:05 2022] GET /identity/v3/services => generated 772 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:58:06.189669 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f117768-a0ce-413e-aa1a-802be65fe061 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:58:06.195592 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-8f117768-a0ce-413e-aa1a-802be65fe061 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 05:58:06.195592 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:58:06.195592 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:58:06.195592 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:58:06.195592 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:58:06.195592 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:58:06.195592 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:58:06.195592 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:58:06.195592 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:58:06.195592 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:58:06.195592 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:58:06.195592 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:58:06.195592 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:58:06.195592 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:58:06.195592 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:58:06.195592 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:58:06.196547 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:58:06.196547 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:58:06.196547 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:58:06.196547 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:58:06.196547 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:58:06.196547 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:58:06.196547 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:58:06.196547 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:58:06.196547 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:58:06.196547 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:58:06.196547 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:58:06.196547 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 24 05:58:06.196547 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:58:06.196547 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:58:06.196547 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:58:06.196547 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:58:06.197382 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:58:06.197382 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:58:06.197382 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:58:06.197382 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:58:06.197382 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:58:06.197382 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:58:06.197382 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:58:06.197382 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:58:06.197382 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:58:06.197382 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:58:06.197382 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 05:58:06.197382 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 24 05:58:06.203018 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 390/785] 10.222.0.62 () {56 vars in 1096 bytes} [Wed Aug 24 05:58:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 378 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:58:06.206164 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4bbbce-13cb-4b6d-88e6-555c64115290 None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:06.250472 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f117768-a0ce-413e-aa1a-802be65fe061 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:06.330877 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4bbbce-13cb-4b6d-88e6-555c64115290 None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:06.377414 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d866fe63-ea8f-421c-9716-06f7f3836c82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:06.378038 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d866fe63-ea8f-421c-9716-06f7f3836c82 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:06.378246 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d866fe63-ea8f-421c-9716-06f7f3836c82 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:06.378505 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d866fe63-ea8f-421c-9716-06f7f3836c82 None admin] PATH_INFO: `/v3/roles` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:06.379187 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d866fe63-ea8f-421c-9716-06f7f3836c82 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:58:06.379493 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d866fe63-ea8f-421c-9716-06f7f3836c82 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:06.385614 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d866fe63-ea8f-421c-9716-06f7f3836c82 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:06.392805 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 396/786] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Aug 24 05:58:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:58:06.399568 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d866fe63-ea8f-421c-9716-06f7f3836c82 None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:06.423550 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f117768-a0ce-413e-aa1a-802be65fe061 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:06.449945 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:06.460829 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25303434-5762-4352-9640-e2197e0974dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:06.461518 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25303434-5762-4352-9640-e2197e0974dc None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:06.465903 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25303434-5762-4352-9640-e2197e0974dc None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:06.465903 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25303434-5762-4352-9640-e2197e0974dc None admin] PATH_INFO: `/v3/roles/service` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:06.473922 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25303434-5762-4352-9640-e2197e0974dc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:58:06.479219 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-25303434-5762-4352-9640-e2197e0974dc None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 05:58:06.479219 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:58:06.479219 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:58:06.479219 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:58:06.479219 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:58:06.479219 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:58:06.479219 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:58:06.479219 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:58:06.479219 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:58:06.479219 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:58:06.479219 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:58:06.479219 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:58:06.479219 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:58:06.479219 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:58:06.479219 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:58:06.479219 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:58:06.480144 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:58:06.480144 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:58:06.480144 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:58:06.480144 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:58:06.480144 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:58:06.480144 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:58:06.480144 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:58:06.480144 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:58:06.480144 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:58:06.480144 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:58:06.480144 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:58:06.480144 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 24 05:58:06.480144 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:58:06.480144 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:58:06.480144 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:58:06.480144 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:58:06.480991 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:58:06.480991 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:58:06.480991 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:58:06.480991 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:58:06.480991 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:58:06.480991 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:58:06.480991 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:58:06.480991 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:58:06.480991 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:58:06.480991 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:58:06.480991 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 05:58:06.480991 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 24 05:58:06.486246 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 391/787] 10.222.0.62 () {56 vars in 1102 bytes} [Wed Aug 24 05:58:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:58:06.487722 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25303434-5762-4352-9640-e2197e0974dc None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:06.514160 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d866fe63-ea8f-421c-9716-06f7f3836c82 None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:06.540157 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:06.553734 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a880c5e-5dce-4a89-9a13-a4c4e882be34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:06.554365 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a880c5e-5dce-4a89-9a13-a4c4e882be34 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:06.554640 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a880c5e-5dce-4a89-9a13-a4c4e882be34 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:06.554908 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a880c5e-5dce-4a89-9a13-a4c4e882be34 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:06.566085 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a880c5e-5dce-4a89-9a13-a4c4e882be34 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:06.567533 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a880c5e-5dce-4a89-9a13-a4c4e882be34 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:06.584252 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application [None req-7a880c5e-5dce-4a89-9a13-a4c4e882be34 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:58:06.584252 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:58:06.584252 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:58:06.584252 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:58:06.584252 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:58:06.584252 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:58:06.584252 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:58:06.584252 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:58:06.584252 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:58:06.584252 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:58:06.584252 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:58:06.584252 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:58:06.584252 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 05:58:06.584252 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 05:58:06.584252 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 05:58:06.584252 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 05:58:06.585270 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:58:06.585270 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:58:06.585270 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:58:06.585270 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:58:06.585270 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:58:06.585270 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:58:06.585270 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:58:06.585270 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application with Lock( Aug 24 05:58:06.585270 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:58:06.585270 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:58:06.585270 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:58:06.585270 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:58:06.585270 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:58:06.585270 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:58:06.585270 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:58:06.585270 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:58:06.585270 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 05:58:06.586739 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 05:58:06.586739 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:58:06.586739 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Aug 24 05:58:06.586739 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 397/788] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Aug 24 05:58:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:58:06.594066 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a880c5e-5dce-4a89-9a13-a4c4e882be34 None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:06.602279 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25303434-5762-4352-9640-e2197e0974dc None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:06.640521 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d60a023-6169-412a-b8cb-26501399eed8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:06.641294 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d60a023-6169-412a-b8cb-26501399eed8 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:06.642043 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d60a023-6169-412a-b8cb-26501399eed8 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:06.642326 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d60a023-6169-412a-b8cb-26501399eed8 None admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:06.643311 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d60a023-6169-412a-b8cb-26501399eed8 None admin] RBAC: Adding query filter params (name=service) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:58:06.643619 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d60a023-6169-412a-b8cb-26501399eed8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:06.645704 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d60a023-6169-412a-b8cb-26501399eed8 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:06.657831 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 392/789] 10.222.0.62 () {56 vars in 1103 bytes} [Wed Aug 24 05:58:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:58:06.663361 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d60a023-6169-412a-b8cb-26501399eed8 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:06.716690 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:06.733076 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a880c5e-5dce-4a89-9a13-a4c4e882be34 None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:06.771789 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f3ae79-3415-42b9-9cab-4fbec1df97d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:06.772680 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f3ae79-3415-42b9-9cab-4fbec1df97d9 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:06.773012 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f3ae79-3415-42b9-9cab-4fbec1df97d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:06.773341 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f3ae79-3415-42b9-9cab-4fbec1df97d9 None admin] PATH_INFO: `/v3/domains` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:06.774173 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f3ae79-3415-42b9-9cab-4fbec1df97d9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:58:06.774597 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f3ae79-3415-42b9-9cab-4fbec1df97d9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:06.776343 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f3ae79-3415-42b9-9cab-4fbec1df97d9 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:06.777687 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d60a023-6169-412a-b8cb-26501399eed8 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:06.788644 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:06.791000 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 398/790] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Aug 24 05:58:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:58:06.796986 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f3ae79-3415-42b9-9cab-4fbec1df97d9 None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:06.816273 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc42e74-e7f6-47cc-86d3-05d62cf43aa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:06.817092 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc42e74-e7f6-47cc-86d3-05d62cf43aa3 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:06.817374 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc42e74-e7f6-47cc-86d3-05d62cf43aa3 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:06.817635 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc42e74-e7f6-47cc-86d3-05d62cf43aa3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:06.829599 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc42e74-e7f6-47cc-86d3-05d62cf43aa3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:06.831166 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc42e74-e7f6-47cc-86d3-05d62cf43aa3 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:06.849078 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-bdc42e74-e7f6-47cc-86d3-05d62cf43aa3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:58:06.849078 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:58:06.849078 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:58:06.849078 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:58:06.849078 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:58:06.849078 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:58:06.849078 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:58:06.849078 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:58:06.849078 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:58:06.849078 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:58:06.849078 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:58:06.849078 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:58:06.849078 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 05:58:06.849078 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 05:58:06.849078 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 05:58:06.849078 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 05:58:06.850101 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:58:06.850101 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:58:06.850101 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:58:06.850101 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:58:06.850101 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:58:06.850101 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:58:06.850101 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:58:06.850101 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 24 05:58:06.850101 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:58:06.850101 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:58:06.850101 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:58:06.850101 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:58:06.850101 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:58:06.850101 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:58:06.850101 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:58:06.850101 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:58:06.850101 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 05:58:06.851007 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 05:58:06.851007 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:58:06.851007 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 24 05:58:06.851007 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 393/791] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Aug 24 05:58:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 05:58:06.856076 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc42e74-e7f6-47cc-86d3-05d62cf43aa3 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:06.909228 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f3ae79-3415-42b9-9cab-4fbec1df97d9 None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:06.946778 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ac194f-8727-4293-a896-1ab237464b54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:06.947779 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ac194f-8727-4293-a896-1ab237464b54 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:06.948272 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ac194f-8727-4293-a896-1ab237464b54 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:06.948799 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ac194f-8727-4293-a896-1ab237464b54 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:06.957869 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ac194f-8727-4293-a896-1ab237464b54 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:06.959588 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ac194f-8727-4293-a896-1ab237464b54 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:06.972098 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application [None req-62ac194f-8727-4293-a896-1ab237464b54 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 24 05:58:06.972098 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:58:06.972098 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:58:06.972098 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:58:06.972098 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:58:06.972098 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:58:06.972098 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:58:06.972098 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:58:06.972098 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:58:06.972098 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:58:06.972098 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:58:06.972098 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:58:06.972098 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 05:58:06.972098 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 05:58:06.972098 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 05:58:06.972098 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 05:58:06.973078 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:58:06.973078 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:58:06.973078 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 05:58:06.973078 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 05:58:06.973078 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 05:58:06.973078 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 05:58:06.973078 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:58:06.973078 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:58:06.973078 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:58:06.973078 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:58:06.973078 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:58:06.973078 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application with Lock( Aug 24 05:58:06.973078 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:58:06.973078 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:58:06.973078 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:58:06.973078 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:58:06.973078 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:58:06.973078 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:58:06.974006 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:58:06.974006 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:58:06.974006 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 05:58:06.974006 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 05:58:06.974006 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 05:58:06.974006 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 05:58:06.974006 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 05:58:06.974006 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 05:58:06.974006 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 24 05:58:06.974006 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Aug 24 05:58:06.976649 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 399/792] 10.222.0.62 () {56 vars in 1096 bytes} [Wed Aug 24 05:58:06 2022] GET /identity/v3/users/swift => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 05:58:06.981050 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc42e74-e7f6-47cc-86d3-05d62cf43aa3 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:06.982568 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee87b2d-85ad-4362-806b-ee022bb38f5a None None] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:06.983112 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee87b2d-85ad-4362-806b-ee022bb38f5a None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:06.983852 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee87b2d-85ad-4362-806b-ee022bb38f5a None None] PATH_INFO: `/` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:06.986406 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 400/793] 10.222.0.62 () {54 vars in 927 bytes} [Wed Aug 24 05:58:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:58:06.987947 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee87b2d-85ad-4362-806b-ee022bb38f5a None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:06.999719 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:07.025212 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef85b093-1ee3-4b1d-ad60-b77a20e6b29e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:07.025926 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef85b093-1ee3-4b1d-ad60-b77a20e6b29e None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:07.026291 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef85b093-1ee3-4b1d-ad60-b77a20e6b29e None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:07.026632 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef85b093-1ee3-4b1d-ad60-b77a20e6b29e None admin] PATH_INFO: `/v3/domains` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:07.027491 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef85b093-1ee3-4b1d-ad60-b77a20e6b29e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:58:07.028625 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef85b093-1ee3-4b1d-ad60-b77a20e6b29e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:07.030101 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef85b093-1ee3-4b1d-ad60-b77a20e6b29e None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:07.050670 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 394/794] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Aug 24 05:58:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:58:07.051917 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a2b931-e236-451f-a3dd-c0b782d9c9cc None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:07.052198 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a2b931-e236-451f-a3dd-c0b782d9c9cc None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:07.052467 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a2b931-e236-451f-a3dd-c0b782d9c9cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:07.074326 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-21a2b931-e236-451f-a3dd-c0b782d9c9cc None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:58:07.075758 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-21a2b931-e236-451f-a3dd-c0b782d9c9cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:58:07.094211 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:07.112019 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee87b2d-85ad-4362-806b-ee022bb38f5a None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:07.153641 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b25ed9f-2fb8-4ceb-80ba-a2de935dd320 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:07.154714 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b25ed9f-2fb8-4ceb-80ba-a2de935dd320 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:07.155321 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b25ed9f-2fb8-4ceb-80ba-a2de935dd320 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:07.155845 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b25ed9f-2fb8-4ceb-80ba-a2de935dd320 None admin] PATH_INFO: `/v3/users` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:07.156870 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b25ed9f-2fb8-4ceb-80ba-a2de935dd320 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:58:07.157447 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b25ed9f-2fb8-4ceb-80ba-a2de935dd320 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:07.159338 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b25ed9f-2fb8-4ceb-80ba-a2de935dd320 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:07.164882 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 395/795] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Aug 24 05:58:07 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 05:58:07.165805 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a2b931-e236-451f-a3dd-c0b782d9c9cc None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:07.181534 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 401/796] 10.222.0.62 () {56 vars in 1135 bytes} [Wed Aug 24 05:58:06 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:58:07.187907 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b25ed9f-2fb8-4ceb-80ba-a2de935dd320 None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:07.287411 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a2b931-e236-451f-a3dd-c0b782d9c9cc None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:07.309763 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:07.330571 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952adbb7-0d99-4100-93f7-c70be969b524 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:07.331439 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952adbb7-0d99-4100-93f7-c70be969b524 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:07.331899 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952adbb7-0d99-4100-93f7-c70be969b524 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:07.332322 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952adbb7-0d99-4100-93f7-c70be969b524 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:07.340605 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952adbb7-0d99-4100-93f7-c70be969b524 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:07.342353 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952adbb7-0d99-4100-93f7-c70be969b524 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:07.358881 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-952adbb7-0d99-4100-93f7-c70be969b524 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 05:58:07.358881 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:58:07.358881 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:58:07.358881 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:58:07.358881 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:58:07.358881 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:58:07.358881 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:58:07.358881 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:58:07.358881 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:58:07.358881 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:58:07.358881 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:58:07.358881 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:58:07.358881 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 05:58:07.358881 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 05:58:07.358881 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 05:58:07.358881 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 05:58:07.359891 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:58:07.359891 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:58:07.359891 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 05:58:07.359891 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 05:58:07.359891 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 05:58:07.359891 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 05:58:07.359891 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:58:07.359891 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:58:07.359891 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:58:07.359891 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:58:07.359891 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:58:07.359891 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 24 05:58:07.359891 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:58:07.359891 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:58:07.359891 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:58:07.359891 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:58:07.359891 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:58:07.359891 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:58:07.360873 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:58:07.360873 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:58:07.360873 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 05:58:07.360873 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 05:58:07.360873 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 05:58:07.360873 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 05:58:07.360873 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 05:58:07.360873 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 05:58:07.360873 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 05:58:07.360873 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 24 05:58:07.363672 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 396/797] 10.222.0.62 () {56 vars in 1117 bytes} [Wed Aug 24 05:58:07 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:58:07.369556 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952adbb7-0d99-4100-93f7-c70be969b524 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:07.384172 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:07.394019 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b25ed9f-2fb8-4ceb-80ba-a2de935dd320 None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:07.453164 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c63c79d-8a38-43b8-b100-b30a5633d203 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:07.454388 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c63c79d-8a38-43b8-b100-b30a5633d203 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:07.455082 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c63c79d-8a38-43b8-b100-b30a5633d203 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:07.455543 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c63c79d-8a38-43b8-b100-b30a5633d203 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:07.470287 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c63c79d-8a38-43b8-b100-b30a5633d203 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:07.472597 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c63c79d-8a38-43b8-b100-b30a5633d203 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:07.491023 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952adbb7-0d99-4100-93f7-c70be969b524 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:07.492666 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application [None req-2c63c79d-8a38-43b8-b100-b30a5633d203 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:58:07.492666 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:58:07.492666 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:58:07.492666 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:58:07.492666 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:58:07.492666 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:58:07.492666 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:58:07.492666 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:58:07.492666 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:58:07.492666 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:58:07.492666 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:58:07.492666 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:58:07.492666 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 05:58:07.492666 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 05:58:07.492666 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 05:58:07.492666 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 05:58:07.493614 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:58:07.493614 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:58:07.493614 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:58:07.493614 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:58:07.493614 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:58:07.493614 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:58:07.493614 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:58:07.493614 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application with Lock( Aug 24 05:58:07.493614 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:58:07.493614 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:58:07.493614 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:58:07.493614 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:58:07.493614 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:58:07.493614 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:58:07.493614 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:58:07.493614 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:58:07.493614 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 05:58:07.494550 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 05:58:07.494550 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:58:07.494550 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Aug 24 05:58:07.497233 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 402/798] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Aug 24 05:58:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 310 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 05:58:07.498370 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c63c79d-8a38-43b8-b100-b30a5633d203 None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:07.540685 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e16bae1-ed0f-4586-a25e-e5b8875696ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:07.541375 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e16bae1-ed0f-4586-a25e-e5b8875696ac None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:07.541653 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e16bae1-ed0f-4586-a25e-e5b8875696ac None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:07.541923 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e16bae1-ed0f-4586-a25e-e5b8875696ac None admin] PATH_INFO: `/v3/users` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:07.542651 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e16bae1-ed0f-4586-a25e-e5b8875696ac None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:58:07.542955 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e16bae1-ed0f-4586-a25e-e5b8875696ac None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:07.544775 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e16bae1-ed0f-4586-a25e-e5b8875696ac None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:07.572548 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 397/799] 10.222.0.62 () {56 vars in 1149 bytes} [Wed Aug 24 05:58:07 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:58:07.572548 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e16bae1-ed0f-4586-a25e-e5b8875696ac None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:07.629247 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c63c79d-8a38-43b8-b100-b30a5633d203 None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:07.644890 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:07.646188 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:07.665132 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd034f7d-4a25-4972-b1ac-f8532afc012c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:07.666184 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd034f7d-4a25-4972-b1ac-f8532afc012c None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:07.666771 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd034f7d-4a25-4972-b1ac-f8532afc012c None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:07.667330 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd034f7d-4a25-4972-b1ac-f8532afc012c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:07.678412 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e16bae1-ed0f-4586-a25e-e5b8875696ac None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:07.681468 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd034f7d-4a25-4972-b1ac-f8532afc012c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:58:07.687070 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application [None req-fd034f7d-4a25-4972-b1ac-f8532afc012c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 05:58:07.687070 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:58:07.687070 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:58:07.687070 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:58:07.687070 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:58:07.687070 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:58:07.687070 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:58:07.687070 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:58:07.687070 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:58:07.687070 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:58:07.687070 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:58:07.687070 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:58:07.687070 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:58:07.687070 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:58:07.687070 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:58:07.687070 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:58:07.691001 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:58:07.691001 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:58:07.691001 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:58:07.691001 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:58:07.691001 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:58:07.691001 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:58:07.691001 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:58:07.691001 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:58:07.691001 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:58:07.691001 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:58:07.691001 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:58:07.691001 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application with Lock( Aug 24 05:58:07.691001 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:58:07.691001 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:58:07.691001 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:58:07.691001 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:58:07.691871 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:58:07.691871 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:58:07.691871 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:58:07.691871 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:58:07.691871 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:58:07.691871 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:58:07.691871 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:58:07.691871 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:58:07.691871 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:58:07.691871 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:58:07.691871 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 05:58:07.691871 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Aug 24 05:58:07.691871 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 403/800] 10.222.0.62 () {56 vars in 1096 bytes} [Wed Aug 24 05:58:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:58:07.691871 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd034f7d-4a25-4972-b1ac-f8532afc012c None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:07.720983 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb31e239-900b-4fdc-89c8-28fd85341a8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:07.721591 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb31e239-900b-4fdc-89c8-28fd85341a8e None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:07.721882 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb31e239-900b-4fdc-89c8-28fd85341a8e None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:07.722123 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb31e239-900b-4fdc-89c8-28fd85341a8e None admin] PATH_INFO: `/v3/domains` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:07.722895 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb31e239-900b-4fdc-89c8-28fd85341a8e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:58:07.723079 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb31e239-900b-4fdc-89c8-28fd85341a8e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:07.724430 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb31e239-900b-4fdc-89c8-28fd85341a8e None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:07.741920 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 398/801] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Aug 24 05:58:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:58:07.743531 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fb8967-2930-4c1a-a7aa-c2a3af0d9e96 None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:07.744015 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fb8967-2930-4c1a-a7aa-c2a3af0d9e96 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:07.744170 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fb8967-2930-4c1a-a7aa-c2a3af0d9e96 None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:07.745637 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 399/802] 10.222.0.62 () {54 vars in 927 bytes} [Wed Aug 24 05:58:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:58:07.746262 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fb8967-2930-4c1a-a7aa-c2a3af0d9e96 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:07.808162 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd034f7d-4a25-4972-b1ac-f8532afc012c None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:07.845474 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d262c00-26e2-49ee-8280-b86c3a5e3449 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:07.846098 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d262c00-26e2-49ee-8280-b86c3a5e3449 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:07.846376 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d262c00-26e2-49ee-8280-b86c3a5e3449 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:07.846647 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d262c00-26e2-49ee-8280-b86c3a5e3449 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:07.857297 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d262c00-26e2-49ee-8280-b86c3a5e3449 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:07.858390 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fb8967-2930-4c1a-a7aa-c2a3af0d9e96 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:07.858762 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d262c00-26e2-49ee-8280-b86c3a5e3449 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:07.872867 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application [None req-4d262c00-26e2-49ee-8280-b86c3a5e3449 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:58:07.872867 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:58:07.872867 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:58:07.872867 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:58:07.872867 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:58:07.872867 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:58:07.872867 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:58:07.872867 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:58:07.872867 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:58:07.872867 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:58:07.872867 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:58:07.872867 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:58:07.872867 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 05:58:07.872867 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 05:58:07.872867 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 05:58:07.872867 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 05:58:07.873828 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:58:07.873828 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:58:07.873828 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:58:07.873828 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:58:07.873828 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:58:07.873828 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:58:07.873828 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:58:07.873828 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application with Lock( Aug 24 05:58:07.873828 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:58:07.873828 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:58:07.873828 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:58:07.873828 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:58:07.873828 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:58:07.873828 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:58:07.873828 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:58:07.873828 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:58:07.873828 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 05:58:07.874712 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 05:58:07.874712 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:58:07.874712 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Aug 24 05:58:07.874712 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 404/803] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Aug 24 05:58:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:58:07.874712 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d262c00-26e2-49ee-8280-b86c3a5e3449 None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:07.892687 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:07.900756 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21419d56-17fc-4507-b1d0-85e9a0b5246d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:07.901343 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21419d56-17fc-4507-b1d0-85e9a0b5246d None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:07.901612 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21419d56-17fc-4507-b1d0-85e9a0b5246d None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:07.901887 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21419d56-17fc-4507-b1d0-85e9a0b5246d None admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:07.902547 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21419d56-17fc-4507-b1d0-85e9a0b5246d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:58:07.902847 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21419d56-17fc-4507-b1d0-85e9a0b5246d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:07.904207 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21419d56-17fc-4507-b1d0-85e9a0b5246d None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:07.906188 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:07.913911 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 400/804] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Aug 24 05:58:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:58:07.914973 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdfaae8-838a-4217-a3cf-d4523d896572 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:07.915254 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdfaae8-838a-4217-a3cf-d4523d896572 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:07.915524 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdfaae8-838a-4217-a3cf-d4523d896572 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:07.933607 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-6bdfaae8-838a-4217-a3cf-d4523d896572 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:58:07.934780 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-6bdfaae8-838a-4217-a3cf-d4523d896572 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:58:07.986169 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d262c00-26e2-49ee-8280-b86c3a5e3449 None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:08.004678 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 401/805] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Aug 24 05:58:07 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 05:58:08.004678 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdfaae8-838a-4217-a3cf-d4523d896572 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:08.035726 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2729841f-ab82-404c-a69f-2341d28a8196 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:08.036432 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2729841f-ab82-404c-a69f-2341d28a8196 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:08.036808 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2729841f-ab82-404c-a69f-2341d28a8196 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:08.037204 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2729841f-ab82-404c-a69f-2341d28a8196 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:08.044482 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2729841f-ab82-404c-a69f-2341d28a8196 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:08.046499 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2729841f-ab82-404c-a69f-2341d28a8196 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:08.060031 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application [None req-2729841f-ab82-404c-a69f-2341d28a8196 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 05:58:08.060031 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:58:08.060031 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:58:08.060031 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:58:08.060031 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:58:08.060031 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:58:08.060031 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:58:08.060031 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:58:08.060031 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:58:08.060031 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:58:08.060031 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:58:08.060031 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:58:08.060031 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 05:58:08.060031 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 05:58:08.060031 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 05:58:08.061174 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 05:58:08.061174 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:58:08.061174 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:58:08.061174 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:58:08.061174 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:58:08.061174 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:58:08.061174 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:58:08.061174 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:58:08.061174 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application with Lock( Aug 24 05:58:08.061174 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:58:08.061174 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:58:08.061174 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:58:08.061174 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:58:08.061174 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:58:08.061174 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:58:08.061174 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:58:08.061174 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:58:08.062260 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 05:58:08.062260 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 05:58:08.062260 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 05:58:08.062260 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 05:58:08.062260 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 05:58:08.062260 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 05:58:08.062260 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 05:58:08.062260 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Aug 24 05:58:08.068133 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 405/806] 10.222.0.62 () {56 vars in 1111 bytes} [Wed Aug 24 05:58:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:58:08.069194 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2729841f-ab82-404c-a69f-2341d28a8196 None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:08.111634 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdfaae8-838a-4217-a3cf-d4523d896572 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:08.133184 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:08.161004 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce7d768-2482-4e49-bef6-5130f19df119 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:08.161616 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce7d768-2482-4e49-bef6-5130f19df119 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:08.161889 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce7d768-2482-4e49-bef6-5130f19df119 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:08.162157 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce7d768-2482-4e49-bef6-5130f19df119 None admin] PATH_INFO: `/v3/domains` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:08.162918 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce7d768-2482-4e49-bef6-5130f19df119 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:58:08.163224 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce7d768-2482-4e49-bef6-5130f19df119 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:08.164997 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce7d768-2482-4e49-bef6-5130f19df119 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:08.177229 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 402/807] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Aug 24 05:58:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:58:08.178123 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce7d768-2482-4e49-bef6-5130f19df119 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:08.179597 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:08.194192 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2729841f-ab82-404c-a69f-2341d28a8196 None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:08.233957 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bdabbd-4fc8-440f-8505-548acf031c23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:08.234704 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bdabbd-4fc8-440f-8505-548acf031c23 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:08.234985 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bdabbd-4fc8-440f-8505-548acf031c23 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:08.235258 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bdabbd-4fc8-440f-8505-548acf031c23 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:08.246513 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5bdabbd-4fc8-440f-8505-548acf031c23 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:08.248006 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5bdabbd-4fc8-440f-8505-548acf031c23 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:08.265110 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application [None req-e5bdabbd-4fc8-440f-8505-548acf031c23 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:58:08.265110 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:58:08.265110 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:58:08.265110 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:58:08.265110 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:58:08.265110 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:58:08.265110 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:58:08.265110 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:58:08.265110 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:58:08.265110 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:58:08.265110 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:58:08.265110 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:58:08.265110 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 05:58:08.265110 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 05:58:08.265110 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 05:58:08.265110 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 05:58:08.266263 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:58:08.266263 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:58:08.266263 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:58:08.266263 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:58:08.266263 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:58:08.266263 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:58:08.266263 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:58:08.266263 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application with Lock( Aug 24 05:58:08.266263 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:58:08.266263 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:58:08.266263 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:58:08.266263 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:58:08.266263 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:58:08.266263 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:58:08.266263 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:58:08.266263 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:58:08.266263 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 05:58:08.267364 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 05:58:08.267364 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:58:08.267364 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Aug 24 05:58:08.270865 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 406/808] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Aug 24 05:58:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:58:08.271809 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bdabbd-4fc8-440f-8505-548acf031c23 None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:08.295739 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce7d768-2482-4e49-bef6-5130f19df119 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:08.330673 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e16de59-2010-4922-b73a-bb99df5b63c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:08.331383 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e16de59-2010-4922-b73a-bb99df5b63c1 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:08.331727 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e16de59-2010-4922-b73a-bb99df5b63c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:08.332060 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e16de59-2010-4922-b73a-bb99df5b63c1 None admin] PATH_INFO: `/v3/projects` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:08.332848 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e16de59-2010-4922-b73a-bb99df5b63c1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:58:08.333220 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e16de59-2010-4922-b73a-bb99df5b63c1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:08.334679 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e16de59-2010-4922-b73a-bb99df5b63c1 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:08.351358 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 403/809] 10.222.0.62 () {56 vars in 1148 bytes} [Wed Aug 24 05:58:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:58:08.351358 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e16de59-2010-4922-b73a-bb99df5b63c1 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:08.379156 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bdabbd-4fc8-440f-8505-548acf031c23 None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:08.394523 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:08.414226 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcfca98-421d-4276-9d9a-cdac4c563de9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:08.414959 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcfca98-421d-4276-9d9a-cdac4c563de9 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:08.415239 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcfca98-421d-4276-9d9a-cdac4c563de9 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:08.415509 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcfca98-421d-4276-9d9a-cdac4c563de9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:08.422796 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dcfca98-421d-4276-9d9a-cdac4c563de9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:08.424228 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dcfca98-421d-4276-9d9a-cdac4c563de9 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:08.427098 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:08.437781 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application [None req-0dcfca98-421d-4276-9d9a-cdac4c563de9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 05:58:08.437781 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:58:08.437781 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:58:08.437781 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:58:08.437781 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:58:08.437781 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:58:08.437781 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:58:08.437781 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:58:08.437781 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:58:08.437781 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:58:08.437781 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:58:08.437781 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:58:08.437781 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 05:58:08.437781 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 05:58:08.437781 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 05:58:08.439589 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 05:58:08.439589 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:58:08.439589 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:58:08.439589 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:58:08.439589 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:58:08.439589 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:58:08.439589 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:58:08.439589 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:58:08.439589 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application with Lock( Aug 24 05:58:08.439589 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:58:08.439589 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:58:08.439589 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:58:08.439589 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:58:08.439589 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:58:08.439589 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:58:08.439589 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:58:08.439589 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:58:08.440526 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 05:58:08.440526 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 05:58:08.440526 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 05:58:08.440526 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 05:58:08.440526 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 05:58:08.440526 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 05:58:08.440526 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 05:58:08.440526 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Aug 24 05:58:08.440526 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 407/810] 10.222.0.62 () {56 vars in 1111 bytes} [Wed Aug 24 05:58:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:58:08.440526 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcfca98-421d-4276-9d9a-cdac4c563de9 None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:08.457581 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e16de59-2010-4922-b73a-bb99df5b63c1 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:08.496675 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3c7d8a-27c6-4d68-84a2-e2d431795adb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:08.497257 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3c7d8a-27c6-4d68-84a2-e2d431795adb None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:08.497535 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3c7d8a-27c6-4d68-84a2-e2d431795adb None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:08.497805 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3c7d8a-27c6-4d68-84a2-e2d431795adb None admin] PATH_INFO: `/v3/domains` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:08.498987 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3c7d8a-27c6-4d68-84a2-e2d431795adb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:58:08.499286 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3c7d8a-27c6-4d68-84a2-e2d431795adb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:08.500668 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3c7d8a-27c6-4d68-84a2-e2d431795adb None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:08.516649 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 404/811] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Aug 24 05:58:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:58:08.518776 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3c7d8a-27c6-4d68-84a2-e2d431795adb None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:08.553946 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcfca98-421d-4276-9d9a-cdac4c563de9 None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:08.591204 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cea043-ebb7-4ed6-afb2-aff866b45e3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:08.591831 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cea043-ebb7-4ed6-afb2-aff866b45e3c None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:08.592109 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cea043-ebb7-4ed6-afb2-aff866b45e3c None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:08.592409 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cea043-ebb7-4ed6-afb2-aff866b45e3c None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:08.593016 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10cea043-ebb7-4ed6-afb2-aff866b45e3c None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:08.594381 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10cea043-ebb7-4ed6-afb2-aff866b45e3c None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:08.602891 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application [None req-10cea043-ebb7-4ed6-afb2-aff866b45e3c None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 05:58:08.602891 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:58:08.602891 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 05:58:08.602891 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 05:58:08.602891 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 05:58:08.602891 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 05:58:08.602891 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 05:58:08.602891 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 05:58:08.602891 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 05:58:08.602891 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Aug 24 05:58:08.602891 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 05:58:08.602891 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Aug 24 05:58:08.602891 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:58:08.602891 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:58:08.602891 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:58:08.602891 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:58:08.602891 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:58:08.603977 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:58:08.603977 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:58:08.603977 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:58:08.603977 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:58:08.603977 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:58:08.603977 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:58:08.603977 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 05:58:08.603977 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 05:58:08.603977 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 05:58:08.603977 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 05:58:08.603977 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:58:08.603977 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:58:08.603977 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:58:08.603977 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:58:08.603977 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:58:08.603977 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:58:08.604874 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:58:08.604874 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application with Lock( Aug 24 05:58:08.604874 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:58:08.604874 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:58:08.604874 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:58:08.604874 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:58:08.604874 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:58:08.604874 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:58:08.604874 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:58:08.604874 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:58:08.604874 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 05:58:08.604874 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 05:58:08.604874 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 05:58:08.604874 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Aug 24 05:58:08.604874 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 408/812] 10.222.0.62 () {56 vars in 1129 bytes} [Wed Aug 24 05:58:08 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:58:08.605650 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cea043-ebb7-4ed6-afb2-aff866b45e3c None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:08.618537 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:08.621510 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3c7d8a-27c6-4d68-84a2-e2d431795adb None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:08.654046 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214a4a68-e850-4890-9339-ac05506e731d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:08.654693 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214a4a68-e850-4890-9339-ac05506e731d None admin] REQUEST_METHOD: `PUT` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:08.654993 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214a4a68-e850-4890-9339-ac05506e731d None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:08.655268 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214a4a68-e850-4890-9339-ac05506e731d None admin] PATH_INFO: `/v3/projects/181f4762037e454489d9552cc603179c/users/14d4fed04fb144d08918bd5d4f808d23/roles/49e47cd157d743f6905f1db07ec878a7` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:08.657165 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214a4a68-e850-4890-9339-ac05506e731d None admin] RBAC: Authorizing `identity:create_grant(project_id=181f4762037e454489d9552cc603179c, user_id=14d4fed04fb144d08918bd5d4f808d23, role_id=49e47cd157d743f6905f1db07ec878a7)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:08.658585 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214a4a68-e850-4890-9339-ac05506e731d None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:08.667184 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:08.705173 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cea043-ebb7-4ed6-afb2-aff866b45e3c None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:08.737105 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d02ef5-9677-47ac-8de0-4a1b035b1124 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:08.737696 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d02ef5-9677-47ac-8de0-4a1b035b1124 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:08.737975 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d02ef5-9677-47ac-8de0-4a1b035b1124 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:08.738244 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d02ef5-9677-47ac-8de0-4a1b035b1124 None admin] PATH_INFO: `/v3/projects` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:08.738925 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d02ef5-9677-47ac-8de0-4a1b035b1124 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:58:08.739224 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d02ef5-9677-47ac-8de0-4a1b035b1124 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:08.740702 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d02ef5-9677-47ac-8de0-4a1b035b1124 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:08.752775 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 409/813] 10.222.0.62 () {56 vars in 1148 bytes} [Wed Aug 24 05:58:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:58:08.753492 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d02ef5-9677-47ac-8de0-4a1b035b1124 None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:08.764077 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 405/814] 10.222.0.62 () {58 vars in 1439 bytes} [Wed Aug 24 05:58:08 2022] PUT /identity/v3/projects/181f4762037e454489d9552cc603179c/users/14d4fed04fb144d08918bd5d4f808d23/roles/49e47cd157d743f6905f1db07ec878a7 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:58:08.765398 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8cc203-b5da-4177-b27e-4b46b64bb87a None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:08.765793 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8cc203-b5da-4177-b27e-4b46b64bb87a None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:08.766068 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8cc203-b5da-4177-b27e-4b46b64bb87a None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:08.767299 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 406/815] 10.222.0.62 () {54 vars in 927 bytes} [Wed Aug 24 05:58:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:58:08.768007 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8cc203-b5da-4177-b27e-4b46b64bb87a None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:08.879378 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d02ef5-9677-47ac-8de0-4a1b035b1124 None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:08.886434 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8cc203-b5da-4177-b27e-4b46b64bb87a None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:08.907396 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:08.921577 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90587765-fa56-491b-b2bd-955e6cbba547 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:08.922205 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90587765-fa56-491b-b2bd-955e6cbba547 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:08.922485 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90587765-fa56-491b-b2bd-955e6cbba547 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:08.922756 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90587765-fa56-491b-b2bd-955e6cbba547 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:08.930217 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90587765-fa56-491b-b2bd-955e6cbba547 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:08.931664 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90587765-fa56-491b-b2bd-955e6cbba547 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:08.933686 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03e514e-4fcb-4a5c-80b3-07e08127aff6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:08.934290 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03e514e-4fcb-4a5c-80b3-07e08127aff6 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:08.934563 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03e514e-4fcb-4a5c-80b3-07e08127aff6 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:08.934831 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03e514e-4fcb-4a5c-80b3-07e08127aff6 None admin] PATH_INFO: `/v3/services` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:08.935447 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03e514e-4fcb-4a5c-80b3-07e08127aff6 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:58:08.935745 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03e514e-4fcb-4a5c-80b3-07e08127aff6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:08.937159 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03e514e-4fcb-4a5c-80b3-07e08127aff6 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:08.941997 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 407/816] 10.222.0.62 () {56 vars in 1124 bytes} [Wed Aug 24 05:58:08 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:58:08.942671 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03e514e-4fcb-4a5c-80b3-07e08127aff6 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:08.943745 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application [None req-90587765-fa56-491b-b2bd-955e6cbba547 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 05:58:08.943745 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:58:08.943745 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:58:08.943745 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:58:08.943745 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:58:08.943745 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:58:08.943745 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:58:08.943745 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:58:08.943745 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:58:08.943745 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:58:08.943745 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:58:08.943745 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:58:08.943745 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 05:58:08.943745 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 05:58:08.943745 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 05:58:08.943745 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 05:58:08.944703 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:58:08.944703 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:58:08.944703 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 05:58:08.944703 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 05:58:08.944703 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 05:58:08.944703 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 05:58:08.944703 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:58:08.944703 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:58:08.944703 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:58:08.944703 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:58:08.944703 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:58:08.944703 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application with Lock( Aug 24 05:58:08.944703 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:58:08.944703 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:58:08.944703 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:58:08.944703 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:58:08.944703 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:58:08.944703 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:58:08.945646 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:58:08.945646 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:58:08.945646 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 05:58:08.945646 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 05:58:08.945646 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 05:58:08.945646 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 05:58:08.945646 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 05:58:08.945646 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 05:58:08.945646 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 05:58:08.945646 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Aug 24 05:58:08.945646 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 410/817] 10.222.0.62 () {56 vars in 1093 bytes} [Wed Aug 24 05:58:08 2022] GET /identity/v3/users/nova => generated 82 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:58:08.946854 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950200e1-aca8-42fb-a22a-a1f187953bcf None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:08.946854 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950200e1-aca8-42fb-a22a-a1f187953bcf None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:08.946854 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950200e1-aca8-42fb-a22a-a1f187953bcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:08.967050 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-950200e1-aca8-42fb-a22a-a1f187953bcf None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:58:08.968799 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-950200e1-aca8-42fb-a22a-a1f187953bcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:58:08.976013 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:09.048607 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 411/818] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Aug 24 05:58:08 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 05:58:09.048607 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950200e1-aca8-42fb-a22a-a1f187953bcf None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:09.055724 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03e514e-4fcb-4a5c-80b3-07e08127aff6 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:09.091159 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79dc45b5-9125-43bc-b7e9-ac75b3dd0bc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:09.091819 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dc45b5-9125-43bc-b7e9-ac75b3dd0bc5 None admin] REQUEST_METHOD: `PUT` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:09.092099 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dc45b5-9125-43bc-b7e9-ac75b3dd0bc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:09.092914 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dc45b5-9125-43bc-b7e9-ac75b3dd0bc5 None admin] PATH_INFO: `/v3/projects/181f4762037e454489d9552cc603179c/users/b82726a12436435cb931b986ef884c2e/roles/5d95a5fc0f324a92905533aa8d34ac15` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:09.107902 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79dc45b5-9125-43bc-b7e9-ac75b3dd0bc5 None admin] RBAC: Authorizing `identity:create_grant(project_id=181f4762037e454489d9552cc603179c, user_id=b82726a12436435cb931b986ef884c2e, role_id=5d95a5fc0f324a92905533aa8d34ac15)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:09.110012 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79dc45b5-9125-43bc-b7e9-ac75b3dd0bc5 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:09.161808 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950200e1-aca8-42fb-a22a-a1f187953bcf None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:09.177381 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:09.195607 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 408/819] 10.222.0.62 () {58 vars in 1439 bytes} [Wed Aug 24 05:58:08 2022] PUT /identity/v3/projects/181f4762037e454489d9552cc603179c/users/b82726a12436435cb931b986ef884c2e/roles/5d95a5fc0f324a92905533aa8d34ac15 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:58:09.197286 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79dc45b5-9125-43bc-b7e9-ac75b3dd0bc5 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:09.197547 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b00f9f-39a4-42ca-9e44-b27734d7b03e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:09.198248 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b00f9f-39a4-42ca-9e44-b27734d7b03e None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:09.198596 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b00f9f-39a4-42ca-9e44-b27734d7b03e None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:09.198877 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b00f9f-39a4-42ca-9e44-b27734d7b03e None admin] PATH_INFO: `/v3/services` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:09.199513 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b00f9f-39a4-42ca-9e44-b27734d7b03e None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:58:09.199816 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b00f9f-39a4-42ca-9e44-b27734d7b03e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:09.201316 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b00f9f-39a4-42ca-9e44-b27734d7b03e None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:09.208030 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 412/820] 10.222.0.62 () {56 vars in 1124 bytes} [Wed Aug 24 05:58:09 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:58:09.217799 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b00f9f-39a4-42ca-9e44-b27734d7b03e None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:09.306330 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:09.345177 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79dc45b5-9125-43bc-b7e9-ac75b3dd0bc5 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:09.383800 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3173f3-7958-4047-a305-b8f6cd1ddcea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:09.384587 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3173f3-7958-4047-a305-b8f6cd1ddcea None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:09.384824 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3173f3-7958-4047-a305-b8f6cd1ddcea None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:09.385091 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3173f3-7958-4047-a305-b8f6cd1ddcea None admin] PATH_INFO: `/v3/users` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:09.385814 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3173f3-7958-4047-a305-b8f6cd1ddcea None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:58:09.386116 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3173f3-7958-4047-a305-b8f6cd1ddcea None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:09.387540 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3173f3-7958-4047-a305-b8f6cd1ddcea None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:09.388167 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b00f9f-39a4-42ca-9e44-b27734d7b03e None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:09.410589 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 409/821] 10.222.0.62 () {56 vars in 1133 bytes} [Wed Aug 24 05:58:09 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:58:09.410988 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3173f3-7958-4047-a305-b8f6cd1ddcea None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:09.434448 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772e87a8-ae3f-4eb6-8991-0e06fc767556 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:09.435193 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772e87a8-ae3f-4eb6-8991-0e06fc767556 None admin] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:09.435486 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772e87a8-ae3f-4eb6-8991-0e06fc767556 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:09.435783 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772e87a8-ae3f-4eb6-8991-0e06fc767556 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:09.436676 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772e87a8-ae3f-4eb6-8991-0e06fc767556 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:09.438045 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772e87a8-ae3f-4eb6-8991-0e06fc767556 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:09.524468 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3173f3-7958-4047-a305-b8f6cd1ddcea None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:09.554773 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:09.560429 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376b4d9e-4d47-45bb-9299-25f0d6821d14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:09.561055 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376b4d9e-4d47-45bb-9299-25f0d6821d14 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:09.561334 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376b4d9e-4d47-45bb-9299-25f0d6821d14 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:09.561658 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376b4d9e-4d47-45bb-9299-25f0d6821d14 None admin] PATH_INFO: `/v3/services/network` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:09.562251 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376b4d9e-4d47-45bb-9299-25f0d6821d14 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:09.563641 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376b4d9e-4d47-45bb-9299-25f0d6821d14 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:09.573974 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-376b4d9e-4d47-45bb-9299-25f0d6821d14 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 05:58:09.573974 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:58:09.573974 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 05:58:09.573974 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 05:58:09.573974 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 05:58:09.573974 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 05:58:09.573974 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 05:58:09.573974 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 05:58:09.573974 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 05:58:09.573974 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 24 05:58:09.573974 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 05:58:09.573974 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 24 05:58:09.573974 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:58:09.573974 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:58:09.573974 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:58:09.573974 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:58:09.573974 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:58:09.574981 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:58:09.574981 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:58:09.574981 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:58:09.574981 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:58:09.574981 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:58:09.574981 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:58:09.574981 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 05:58:09.574981 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 05:58:09.574981 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 05:58:09.574981 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 05:58:09.574981 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:58:09.574981 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:58:09.574981 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:58:09.574981 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:58:09.574981 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:58:09.574981 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:58:09.576462 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:58:09.576462 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 24 05:58:09.576462 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:58:09.576462 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:58:09.576462 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:58:09.576462 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:58:09.576462 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:58:09.576462 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:58:09.576462 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:58:09.576462 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:58:09.576462 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 05:58:09.576462 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 05:58:09.576462 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 05:58:09.576462 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 24 05:58:09.580594 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 410/822] 10.222.0.62 () {56 vars in 1111 bytes} [Wed Aug 24 05:58:09 2022] GET /identity/v3/services/network => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:58:09.581279 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376b4d9e-4d47-45bb-9299-25f0d6821d14 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:09.596434 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 413/823] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Aug 24 05:58:09 2022] POST /identity/v3/endpoints => generated 346 bytes in 373 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:58:09.597856 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772e87a8-ae3f-4eb6-8991-0e06fc767556 None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:09.687697 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:09.774794 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772e87a8-ae3f-4eb6-8991-0e06fc767556 None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:09.781902 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376b4d9e-4d47-45bb-9299-25f0d6821d14 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:09.812915 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625a8e76-489d-4d01-8d29-a6ab8b3e1334 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:09.813449 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625a8e76-489d-4d01-8d29-a6ab8b3e1334 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:09.813724 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625a8e76-489d-4d01-8d29-a6ab8b3e1334 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:09.813993 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625a8e76-489d-4d01-8d29-a6ab8b3e1334 None admin] PATH_INFO: `/v3/services` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:09.814698 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625a8e76-489d-4d01-8d29-a6ab8b3e1334 None admin] RBAC: Adding query filter params (name=network) {{(pid=81851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:58:09.814995 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625a8e76-489d-4d01-8d29-a6ab8b3e1334 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:09.816367 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625a8e76-489d-4d01-8d29-a6ab8b3e1334 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:09.825545 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc25faf-cc90-40e7-8608-65f1ee298374 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:09.826160 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc25faf-cc90-40e7-8608-65f1ee298374 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:09.826443 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc25faf-cc90-40e7-8608-65f1ee298374 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:09.827326 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc25faf-cc90-40e7-8608-65f1ee298374 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:09.828176 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 414/824] 10.222.0.62 () {56 vars in 1112 bytes} [Wed Aug 24 05:58:09 2022] GET /identity/v3/services?name=network => generated 124 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:58:09.833844 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625a8e76-489d-4d01-8d29-a6ab8b3e1334 None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:09.911040 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:09.919085 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc25faf-cc90-40e7-8608-65f1ee298374 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:09.920274 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc25faf-cc90-40e7-8608-65f1ee298374 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:09.946197 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:09.954514 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625a8e76-489d-4d01-8d29-a6ab8b3e1334 None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:09.991055 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b2604b-8fab-4109-930a-88ae535bc9ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:09.992052 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b2604b-8fab-4109-930a-88ae535bc9ca None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:09.992610 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b2604b-8fab-4109-930a-88ae535bc9ca None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:09.993093 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b2604b-8fab-4109-930a-88ae535bc9ca None admin] PATH_INFO: `/v3/services` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:09.993996 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b2604b-8fab-4109-930a-88ae535bc9ca None admin] RBAC: Adding query filter params (type=network) {{(pid=81851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:58:09.994491 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b2604b-8fab-4109-930a-88ae535bc9ca None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:09.996037 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b2604b-8fab-4109-930a-88ae535bc9ca None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:10.002599 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 415/825] 10.222.0.62 () {56 vars in 1112 bytes} [Wed Aug 24 05:58:09 2022] GET /identity/v3/services?type=network => generated 350 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:58:10.008274 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b2604b-8fab-4109-930a-88ae535bc9ca None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:10.031729 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 411/826] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Aug 24 05:58:09 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 450 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 05:58:10.037256 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc25faf-cc90-40e7-8608-65f1ee298374 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:10.140643 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b2604b-8fab-4109-930a-88ae535bc9ca None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:10.184863 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:10.189612 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc25faf-cc90-40e7-8608-65f1ee298374 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:10.204773 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1576c1-c504-4c49-9aff-6ddbdbf032e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:10.205385 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1576c1-c504-4c49-9aff-6ddbdbf032e8 None admin] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:10.205663 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1576c1-c504-4c49-9aff-6ddbdbf032e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:10.205935 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1576c1-c504-4c49-9aff-6ddbdbf032e8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:10.206589 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1576c1-c504-4c49-9aff-6ddbdbf032e8 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:10.207962 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1576c1-c504-4c49-9aff-6ddbdbf032e8 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:10.244824 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2b1a57-5316-4645-baed-5e9770cc644e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:10.246961 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2b1a57-5316-4645-baed-5e9770cc644e None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:10.246961 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2b1a57-5316-4645-baed-5e9770cc644e None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:10.246961 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2b1a57-5316-4645-baed-5e9770cc644e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:10.257202 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2b1a57-5316-4645-baed-5e9770cc644e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:10.258797 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2b1a57-5316-4645-baed-5e9770cc644e None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:10.273953 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-1d2b1a57-5316-4645-baed-5e9770cc644e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:58:10.273953 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:58:10.273953 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:58:10.273953 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:58:10.273953 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:58:10.273953 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:58:10.273953 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:58:10.273953 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:58:10.273953 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:58:10.273953 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:58:10.273953 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:58:10.273953 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:58:10.273953 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 05:58:10.273953 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 05:58:10.273953 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 05:58:10.273953 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 05:58:10.275097 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:58:10.275097 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:58:10.275097 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:58:10.275097 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:58:10.275097 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:58:10.275097 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:58:10.275097 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:58:10.275097 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 24 05:58:10.275097 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:58:10.275097 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:58:10.275097 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:58:10.275097 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:58:10.275097 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:58:10.275097 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:58:10.275097 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:58:10.275097 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:58:10.275097 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 05:58:10.276246 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 05:58:10.276246 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:58:10.276246 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 24 05:58:10.276246 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 412/827] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Aug 24 05:58:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 05:58:10.285411 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2b1a57-5316-4645-baed-5e9770cc644e None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:10.314157 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 416/828] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Aug 24 05:58:10 2022] POST /identity/v3/endpoints => generated 337 bytes in 306 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 05:58:10.405767 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2b1a57-5316-4645-baed-5e9770cc644e None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:10.458013 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da27cd2-2e0e-4eff-bf73-6a56d876c1e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:10.458636 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da27cd2-2e0e-4eff-bf73-6a56d876c1e0 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:10.458909 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da27cd2-2e0e-4eff-bf73-6a56d876c1e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:10.459174 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da27cd2-2e0e-4eff-bf73-6a56d876c1e0 None admin] PATH_INFO: `/v3/domains` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:10.459821 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da27cd2-2e0e-4eff-bf73-6a56d876c1e0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:58:10.460118 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da27cd2-2e0e-4eff-bf73-6a56d876c1e0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:10.461528 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da27cd2-2e0e-4eff-bf73-6a56d876c1e0 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:10.475197 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 413/829] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Aug 24 05:58:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:58:10.480903 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1576c1-c504-4c49-9aff-6ddbdbf032e8 None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:10.512857 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:10.603837 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1576c1-c504-4c49-9aff-6ddbdbf032e8 None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:10.638531 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ddb94a-1cc0-4ffe-8d29-d470dcf94c90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:10.639194 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ddb94a-1cc0-4ffe-8d29-d470dcf94c90 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:10.639471 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ddb94a-1cc0-4ffe-8d29-d470dcf94c90 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:10.639742 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ddb94a-1cc0-4ffe-8d29-d470dcf94c90 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:10.646431 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ddb94a-1cc0-4ffe-8d29-d470dcf94c90 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:10.647861 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ddb94a-1cc0-4ffe-8d29-d470dcf94c90 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:10.659441 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application [None req-27ddb94a-1cc0-4ffe-8d29-d470dcf94c90 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 05:58:10.659441 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:58:10.659441 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:58:10.659441 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:58:10.659441 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:58:10.659441 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:58:10.659441 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:58:10.659441 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:58:10.659441 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:58:10.659441 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:58:10.659441 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:58:10.659441 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:58:10.659441 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 05:58:10.659441 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 05:58:10.659441 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 05:58:10.660302 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 05:58:10.660302 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:58:10.660302 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:58:10.660302 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:58:10.660302 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:58:10.660302 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:58:10.660302 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:58:10.660302 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:58:10.660302 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application with Lock( Aug 24 05:58:10.660302 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:58:10.660302 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:58:10.660302 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:58:10.660302 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:58:10.660302 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:58:10.660302 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:58:10.660302 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:58:10.660302 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:58:10.661694 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 05:58:10.661694 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 05:58:10.661694 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 05:58:10.661694 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 05:58:10.661694 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 05:58:10.661694 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 05:58:10.661694 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 05:58:10.661694 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Aug 24 05:58:10.661694 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 417/830] 10.222.0.62 () {56 vars in 1111 bytes} [Wed Aug 24 05:58:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:58:10.667541 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da27cd2-2e0e-4eff-bf73-6a56d876c1e0 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:10.672274 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:10.771697 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da27cd2-2e0e-4eff-bf73-6a56d876c1e0 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:10.806318 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612f6f19-45dc-497e-b01e-8be22ef30841 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:10.806929 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612f6f19-45dc-497e-b01e-8be22ef30841 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:10.807363 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612f6f19-45dc-497e-b01e-8be22ef30841 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:10.807492 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612f6f19-45dc-497e-b01e-8be22ef30841 None admin] PATH_INFO: `/v3/projects` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:10.808138 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612f6f19-45dc-497e-b01e-8be22ef30841 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:58:10.808413 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612f6f19-45dc-497e-b01e-8be22ef30841 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:10.809869 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612f6f19-45dc-497e-b01e-8be22ef30841 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:10.822429 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 414/831] 10.222.0.62 () {56 vars in 1148 bytes} [Wed Aug 24 05:58:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:58:10.828784 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ddb94a-1cc0-4ffe-8d29-d470dcf94c90 None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:10.887683 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb80840-3fa1-49cf-a0bd-fbe84b64d310 None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:10.888001 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb80840-3fa1-49cf-a0bd-fbe84b64d310 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:10.888223 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb80840-3fa1-49cf-a0bd-fbe84b64d310 None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:10.890985 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 415/832] 10.222.0.62 () {54 vars in 927 bytes} [Wed Aug 24 05:58:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 05:58:10.897400 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e0d794-0249-48f1-9af7-d71416d0abdc None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:10.897686 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e0d794-0249-48f1-9af7-d71416d0abdc None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:10.897948 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e0d794-0249-48f1-9af7-d71416d0abdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:10.913601 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:10.919569 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-a8e0d794-0249-48f1-9af7-d71416d0abdc None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:58:10.921115 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-a8e0d794-0249-48f1-9af7-d71416d0abdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:58:10.955937 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ddb94a-1cc0-4ffe-8d29-d470dcf94c90 None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:10.985809 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:10.995671 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 416/833] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Aug 24 05:58:10 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 05:58:10.998041 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac3be40-da9b-4dad-9e4f-abf65739f665 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:10.998678 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac3be40-da9b-4dad-9e4f-abf65739f665 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:10.998952 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac3be40-da9b-4dad-9e4f-abf65739f665 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:10.999220 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac3be40-da9b-4dad-9e4f-abf65739f665 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:10.999935 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac3be40-da9b-4dad-9e4f-abf65739f665 None admin] RBAC: Adding query filter params (role.id=49e47cd157d743f6905f1db07ec878a7, scope.project.id=181f4762037e454489d9552cc603179c, user.id=fb6d3743340241f98cacb9e0d063298c) {{(pid=81851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:58:11.000237 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac3be40-da9b-4dad-9e4f-abf65739f665 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:11.008528 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac3be40-da9b-4dad-9e4f-abf65739f665 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:11.008528 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 418/834] 10.222.0.62 () {56 vars in 1374 bytes} [Wed Aug 24 05:58:10 2022] GET /identity/v3/role_assignments?user.id=fb6d3743340241f98cacb9e0d063298c&scope.project.id=181f4762037e454489d9552cc603179c&role.id=49e47cd157d743f6905f1db07ec878a7 => generated 608 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:58:11.268274 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e0d794-0249-48f1-9af7-d71416d0abdc None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:11.366618 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e0d794-0249-48f1-9af7-d71416d0abdc None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:11.401646 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23df061-bb14-4a70-90a5-273ba2ebbd8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:11.402279 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23df061-bb14-4a70-90a5-273ba2ebbd8f None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:11.402557 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23df061-bb14-4a70-90a5-273ba2ebbd8f None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:11.402828 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23df061-bb14-4a70-90a5-273ba2ebbd8f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:11.413241 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23df061-bb14-4a70-90a5-273ba2ebbd8f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:58:11.416135 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59da4a06-ae5a-48c4-95f3-d939a14188df None None] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:11.416668 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59da4a06-ae5a-48c4-95f3-d939a14188df None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:11.417184 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59da4a06-ae5a-48c4-95f3-d939a14188df None None] PATH_INFO: `/` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:11.418273 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-c23df061-bb14-4a70-90a5-273ba2ebbd8f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 05:58:11.418273 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:58:11.418273 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:58:11.418273 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:58:11.418273 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:58:11.418273 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:58:11.418273 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:58:11.418273 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:58:11.418273 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:58:11.418273 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:58:11.418273 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:58:11.418273 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:58:11.418273 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:58:11.418273 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:58:11.418273 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:58:11.418273 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:58:11.419221 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:58:11.419221 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:58:11.419221 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:58:11.419221 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:58:11.419221 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:58:11.419221 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:58:11.419221 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:58:11.419221 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:58:11.419221 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:58:11.419221 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:58:11.419221 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:58:11.419221 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 24 05:58:11.419221 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:58:11.419221 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:58:11.419221 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:58:11.419221 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:58:11.420387 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:58:11.420387 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:58:11.420387 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:58:11.420387 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:58:11.420387 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:58:11.420387 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:58:11.420387 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:58:11.420387 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:58:11.420387 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:58:11.420387 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:58:11.420387 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 05:58:11.420387 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 24 05:58:11.420387 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 417/835] 10.222.0.62 () {56 vars in 1096 bytes} [Wed Aug 24 05:58:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:58:11.422395 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 419/836] 10.222.0.62 () {54 vars in 927 bytes} [Wed Aug 24 05:58:11 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:58:11.426367 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23df061-bb14-4a70-90a5-273ba2ebbd8f None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:11.427891 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c86609-3ffe-40a3-bfd9-1bcc1727c286 None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:11.428392 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c86609-3ffe-40a3-bfd9-1bcc1727c286 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:11.428911 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c86609-3ffe-40a3-bfd9-1bcc1727c286 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:11.448070 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-f1c86609-3ffe-40a3-bfd9-1bcc1727c286 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:58:11.449683 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-f1c86609-3ffe-40a3-bfd9-1bcc1727c286 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:58:11.494660 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:11.540901 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 420/837] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Aug 24 05:58:11 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 05:58:11.547624 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23df061-bb14-4a70-90a5-273ba2ebbd8f None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:11.583103 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504aa559-0dc6-4b2b-80a0-edee8976ab5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:11.583707 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504aa559-0dc6-4b2b-80a0-edee8976ab5e None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:11.584014 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504aa559-0dc6-4b2b-80a0-edee8976ab5e None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:11.584295 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504aa559-0dc6-4b2b-80a0-edee8976ab5e None admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:11.584975 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504aa559-0dc6-4b2b-80a0-edee8976ab5e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:58:11.585271 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504aa559-0dc6-4b2b-80a0-edee8976ab5e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:11.586618 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504aa559-0dc6-4b2b-80a0-edee8976ab5e None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:11.595417 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 418/838] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Aug 24 05:58:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:58:11.600681 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c86609-3ffe-40a3-bfd9-1bcc1727c286 None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:11.671578 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:11.706698 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c86609-3ffe-40a3-bfd9-1bcc1727c286 None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:11.745341 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca39571d-ffc0-4e34-8202-becb224a4c52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:11.746391 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca39571d-ffc0-4e34-8202-becb224a4c52 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:11.746873 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca39571d-ffc0-4e34-8202-becb224a4c52 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:11.747401 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca39571d-ffc0-4e34-8202-becb224a4c52 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:11.758323 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca39571d-ffc0-4e34-8202-becb224a4c52 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:11.760027 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca39571d-ffc0-4e34-8202-becb224a4c52 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:11.774776 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application [None req-ca39571d-ffc0-4e34-8202-becb224a4c52 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:58:11.774776 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:58:11.774776 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:58:11.774776 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:58:11.774776 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:58:11.774776 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:58:11.774776 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:58:11.774776 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:58:11.774776 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:58:11.774776 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:58:11.774776 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:58:11.774776 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:58:11.774776 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 05:58:11.774776 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 05:58:11.774776 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 05:58:11.774776 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 05:58:11.775719 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:58:11.775719 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:58:11.775719 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:58:11.775719 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:58:11.775719 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:58:11.775719 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:58:11.775719 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:58:11.775719 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application with Lock( Aug 24 05:58:11.775719 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:58:11.775719 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:58:11.775719 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:58:11.775719 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:58:11.775719 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:58:11.775719 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:58:11.775719 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:58:11.775719 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:58:11.775719 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 05:58:11.776658 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 05:58:11.776658 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:58:11.776658 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Aug 24 05:58:11.778334 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 421/839] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Aug 24 05:58:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:58:11.784910 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504aa559-0dc6-4b2b-80a0-edee8976ab5e None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:11.814563 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca39571d-ffc0-4e34-8202-becb224a4c52 None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:11.907093 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504aa559-0dc6-4b2b-80a0-edee8976ab5e None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:11.927128 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:11.928457 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca39571d-ffc0-4e34-8202-becb224a4c52 None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:11.948229 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ecc9ff-023f-4cc8-b492-b5f04ba39a68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:11.948941 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ecc9ff-023f-4cc8-b492-b5f04ba39a68 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:11.949215 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ecc9ff-023f-4cc8-b492-b5f04ba39a68 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:11.949478 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ecc9ff-023f-4cc8-b492-b5f04ba39a68 None admin] PATH_INFO: `/v3/domains` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:11.950112 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ecc9ff-023f-4cc8-b492-b5f04ba39a68 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:58:11.950674 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:11.950881 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ecc9ff-023f-4cc8-b492-b5f04ba39a68 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:11.953108 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ecc9ff-023f-4cc8-b492-b5f04ba39a68 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:11.966026 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 419/840] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Aug 24 05:58:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:58:11.966683 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc862e7-b79e-4299-8e1b-9bc7174dc77a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:11.967274 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d65c66a-a9c7-44a8-a95d-9e7ed7f3b3a2 None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:11.967551 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d65c66a-a9c7-44a8-a95d-9e7ed7f3b3a2 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:11.967817 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d65c66a-a9c7-44a8-a95d-9e7ed7f3b3a2 None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:11.968316 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc862e7-b79e-4299-8e1b-9bc7174dc77a None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:11.968924 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc862e7-b79e-4299-8e1b-9bc7174dc77a None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:11.973201 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 420/841] 10.222.0.62 () {54 vars in 927 bytes} [Wed Aug 24 05:58:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:58:11.973201 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d65c66a-a9c7-44a8-a95d-9e7ed7f3b3a2 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:11.973806 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc862e7-b79e-4299-8e1b-9bc7174dc77a None admin] PATH_INFO: `/v3/roles/service` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:11.985465 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc862e7-b79e-4299-8e1b-9bc7174dc77a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:58:11.990291 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application [None req-fdc862e7-b79e-4299-8e1b-9bc7174dc77a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 05:58:11.990291 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:58:11.990291 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:58:11.990291 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:58:11.990291 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:58:11.990291 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:58:11.990291 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:58:11.990291 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:58:11.990291 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:58:11.990291 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:58:11.990291 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:58:11.990291 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:58:11.990291 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:58:11.990291 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:58:11.990291 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:58:11.990291 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:58:11.991227 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:58:11.991227 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:58:11.991227 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:58:11.991227 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:58:11.991227 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:58:11.991227 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:58:11.991227 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:58:11.991227 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:58:11.991227 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:58:11.991227 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:58:11.991227 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:58:11.991227 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application with Lock( Aug 24 05:58:11.991227 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:58:11.991227 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:58:11.991227 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:58:11.991227 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:58:11.992074 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:58:11.992074 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:58:11.992074 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:58:11.992074 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:58:11.992074 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:58:11.992074 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:58:11.992074 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:58:11.992074 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:58:11.992074 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:58:11.992074 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:58:11.992074 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 05:58:11.992074 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Aug 24 05:58:11.993555 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 422/842] 10.222.0.62 () {56 vars in 1102 bytes} [Wed Aug 24 05:58:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 05:58:11.994221 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476ae6fd-ef81-4a4b-87ad-4f137dc1e1c6 None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:11.994508 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476ae6fd-ef81-4a4b-87ad-4f137dc1e1c6 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:11.994778 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476ae6fd-ef81-4a4b-87ad-4f137dc1e1c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:12.013335 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-476ae6fd-ef81-4a4b-87ad-4f137dc1e1c6 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:58:12.015136 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-476ae6fd-ef81-4a4b-87ad-4f137dc1e1c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:58:12.083350 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d65c66a-a9c7-44a8-a95d-9e7ed7f3b3a2 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:12.084364 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 423/843] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Aug 24 05:58:11 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 05:58:12.089393 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476ae6fd-ef81-4a4b-87ad-4f137dc1e1c6 None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:12.119717 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c492bc78-1825-4ced-9024-da099bbfe3a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:12.120352 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c492bc78-1825-4ced-9024-da099bbfe3a6 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:12.120681 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c492bc78-1825-4ced-9024-da099bbfe3a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:12.120915 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c492bc78-1825-4ced-9024-da099bbfe3a6 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:12.129247 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c492bc78-1825-4ced-9024-da099bbfe3a6 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:12.130746 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c492bc78-1825-4ced-9024-da099bbfe3a6 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:12.143092 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-c492bc78-1825-4ced-9024-da099bbfe3a6 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 24 05:58:12.143092 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:58:12.143092 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:58:12.143092 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:58:12.143092 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:58:12.143092 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:58:12.143092 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:58:12.143092 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:58:12.143092 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:58:12.143092 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:58:12.143092 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:58:12.143092 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:58:12.143092 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 05:58:12.143092 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 05:58:12.143092 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 05:58:12.143092 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 05:58:12.144072 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:58:12.144072 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:58:12.144072 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 05:58:12.144072 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 05:58:12.144072 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 05:58:12.144072 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 05:58:12.144072 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:58:12.144072 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:58:12.144072 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:58:12.144072 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:58:12.144072 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:58:12.144072 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 24 05:58:12.144072 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:58:12.144072 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:58:12.144072 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:58:12.144072 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:58:12.144072 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:58:12.144072 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:58:12.145067 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:58:12.145067 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:58:12.145067 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 05:58:12.145067 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 05:58:12.145067 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 05:58:12.145067 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 05:58:12.145067 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 05:58:12.145067 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 05:58:12.145067 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 24 05:58:12.145067 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 24 05:58:12.149066 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 421/844] 10.222.0.62 () {56 vars in 1096 bytes} [Wed Aug 24 05:58:11 2022] GET /identity/v3/users/swift => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:58:12.149899 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c492bc78-1825-4ced-9024-da099bbfe3a6 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:12.196060 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:12.222408 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476ae6fd-ef81-4a4b-87ad-4f137dc1e1c6 None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:12.237526 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:12.262813 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab128785-45f4-4309-bf5f-0f903ae34c40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:12.263419 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab128785-45f4-4309-bf5f-0f903ae34c40 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:12.263748 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab128785-45f4-4309-bf5f-0f903ae34c40 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:12.263985 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab128785-45f4-4309-bf5f-0f903ae34c40 None admin] PATH_INFO: `/v3/roles` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:12.265203 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab128785-45f4-4309-bf5f-0f903ae34c40 None admin] RBAC: Adding query filter params (name=service) {{(pid=81851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:58:12.265690 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab128785-45f4-4309-bf5f-0f903ae34c40 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:12.268153 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab128785-45f4-4309-bf5f-0f903ae34c40 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:12.282342 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 424/845] 10.222.0.62 () {56 vars in 1103 bytes} [Wed Aug 24 05:58:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:58:12.283152 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab128785-45f4-4309-bf5f-0f903ae34c40 None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:12.285435 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c492bc78-1825-4ced-9024-da099bbfe3a6 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:12.334412 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2e5490-62f3-446d-9f96-da5e4fb26d55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:12.335424 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2e5490-62f3-446d-9f96-da5e4fb26d55 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:12.336317 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2e5490-62f3-446d-9f96-da5e4fb26d55 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:12.336767 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2e5490-62f3-446d-9f96-da5e4fb26d55 None admin] PATH_INFO: `/v3/users` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:12.337908 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2e5490-62f3-446d-9f96-da5e4fb26d55 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:58:12.338628 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2e5490-62f3-446d-9f96-da5e4fb26d55 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:12.340607 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2e5490-62f3-446d-9f96-da5e4fb26d55 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:12.361608 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 422/846] 10.222.0.62 () {56 vars in 1135 bytes} [Wed Aug 24 05:58:12 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:58:12.361990 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2e5490-62f3-446d-9f96-da5e4fb26d55 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:12.397404 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab128785-45f4-4309-bf5f-0f903ae34c40 None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:12.432470 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090bca13-18b9-41a8-bb7e-366667f1f677 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:12.433126 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090bca13-18b9-41a8-bb7e-366667f1f677 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:12.433408 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090bca13-18b9-41a8-bb7e-366667f1f677 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:12.433684 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090bca13-18b9-41a8-bb7e-366667f1f677 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:12.445391 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090bca13-18b9-41a8-bb7e-366667f1f677 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:12.446833 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090bca13-18b9-41a8-bb7e-366667f1f677 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:12.457792 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:12.461510 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application [None req-090bca13-18b9-41a8-bb7e-366667f1f677 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:58:12.461510 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:58:12.461510 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:58:12.461510 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:58:12.461510 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:58:12.461510 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:58:12.461510 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:58:12.461510 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:58:12.461510 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:58:12.461510 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:58:12.461510 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:58:12.461510 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:58:12.461510 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 05:58:12.461510 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 05:58:12.461510 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 05:58:12.461510 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 05:58:12.462438 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:58:12.462438 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:58:12.462438 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:58:12.462438 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:58:12.462438 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:58:12.462438 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:58:12.462438 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:58:12.462438 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application with Lock( Aug 24 05:58:12.462438 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:58:12.462438 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:58:12.462438 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:58:12.462438 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:58:12.462438 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:58:12.462438 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:58:12.462438 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:58:12.462438 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:58:12.462438 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 05:58:12.463339 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 05:58:12.463339 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:58:12.463339 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Aug 24 05:58:12.463339 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 425/847] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Aug 24 05:58:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:58:12.463925 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090bca13-18b9-41a8-bb7e-366667f1f677 None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:12.469698 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2e5490-62f3-446d-9f96-da5e4fb26d55 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:12.495123 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:12.508193 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc66e49-4cc6-4c80-9bea-d82065dd91ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:12.508852 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc66e49-4cc6-4c80-9bea-d82065dd91ad None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:12.509129 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc66e49-4cc6-4c80-9bea-d82065dd91ad None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:12.509396 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc66e49-4cc6-4c80-9bea-d82065dd91ad None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:12.509967 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc66e49-4cc6-4c80-9bea-d82065dd91ad None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:12.511415 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc66e49-4cc6-4c80-9bea-d82065dd91ad None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:12.520386 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-4dc66e49-4cc6-4c80-9bea-d82065dd91ad None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 05:58:12.520386 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:58:12.520386 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 05:58:12.520386 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 05:58:12.520386 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 05:58:12.520386 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 05:58:12.520386 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 05:58:12.520386 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 05:58:12.520386 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 05:58:12.520386 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 24 05:58:12.520386 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 05:58:12.520386 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 24 05:58:12.520386 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:58:12.520386 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:58:12.520386 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:58:12.520386 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:58:12.520386 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:58:12.524681 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:58:12.524681 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:58:12.524681 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:58:12.524681 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:58:12.524681 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:58:12.524681 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:58:12.524681 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 05:58:12.524681 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 05:58:12.524681 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 05:58:12.524681 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 05:58:12.524681 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:58:12.524681 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:58:12.524681 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:58:12.524681 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:58:12.524681 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:58:12.524681 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:58:12.525593 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:58:12.525593 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 24 05:58:12.525593 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:58:12.525593 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:58:12.525593 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:58:12.525593 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:58:12.525593 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:58:12.525593 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:58:12.525593 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:58:12.525593 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:58:12.525593 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 05:58:12.525593 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 05:58:12.525593 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 05:58:12.525593 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 24 05:58:12.525593 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 423/848] 10.222.0.62 () {56 vars in 1114 bytes} [Wed Aug 24 05:58:12 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:58:12.526379 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc66e49-4cc6-4c80-9bea-d82065dd91ad None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:12.583436 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090bca13-18b9-41a8-bb7e-366667f1f677 None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:12.627494 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34741ef8-add6-4f44-a1c0-6a756e2d3c2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:12.627918 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc66e49-4cc6-4c80-9bea-d82065dd91ad None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:12.628031 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34741ef8-add6-4f44-a1c0-6a756e2d3c2c None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:12.628298 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34741ef8-add6-4f44-a1c0-6a756e2d3c2c None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:12.628659 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34741ef8-add6-4f44-a1c0-6a756e2d3c2c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:12.662911 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382ac35c-6b2a-43b3-a171-827fc650278b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:12.663514 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382ac35c-6b2a-43b3-a171-827fc650278b None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:12.663792 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382ac35c-6b2a-43b3-a171-827fc650278b None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:12.664063 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382ac35c-6b2a-43b3-a171-827fc650278b None admin] PATH_INFO: `/v3/domains` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:12.664718 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382ac35c-6b2a-43b3-a171-827fc650278b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:58:12.665020 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382ac35c-6b2a-43b3-a171-827fc650278b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:12.666375 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382ac35c-6b2a-43b3-a171-827fc650278b None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:12.684328 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 424/849] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Aug 24 05:58:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:58:12.684328 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382ac35c-6b2a-43b3-a171-827fc650278b None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:12.706434 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:12.714046 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34741ef8-add6-4f44-a1c0-6a756e2d3c2c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:12.715645 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34741ef8-add6-4f44-a1c0-6a756e2d3c2c None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:12.742184 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:12.790202 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382ac35c-6b2a-43b3-a171-827fc650278b None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:12.818372 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 426/850] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Aug 24 05:58:12 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 05:58:12.823887 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34741ef8-add6-4f44-a1c0-6a756e2d3c2c None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:12.825310 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa84025e-21d1-4bde-99ad-4400e89135ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:12.825916 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa84025e-21d1-4bde-99ad-4400e89135ff None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:12.826197 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa84025e-21d1-4bde-99ad-4400e89135ff None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:12.826495 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa84025e-21d1-4bde-99ad-4400e89135ff None admin] PATH_INFO: `/v3/services` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:12.827115 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa84025e-21d1-4bde-99ad-4400e89135ff None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:58:12.827417 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa84025e-21d1-4bde-99ad-4400e89135ff None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:12.828814 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa84025e-21d1-4bde-99ad-4400e89135ff None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:12.833269 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 425/851] 10.222.0.62 () {56 vars in 1114 bytes} [Wed Aug 24 05:58:12 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:58:12.834032 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa84025e-21d1-4bde-99ad-4400e89135ff None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:12.917019 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:12.934905 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34741ef8-add6-4f44-a1c0-6a756e2d3c2c None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:12.936927 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa84025e-21d1-4bde-99ad-4400e89135ff None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:12.961059 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:12.971887 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54477f89-abcc-4577-8c18-58fc651c52ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:12.972141 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b51c31-1c3f-4889-a29b-5336b292b3cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:12.972517 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54477f89-abcc-4577-8c18-58fc651c52ff None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:12.972692 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b51c31-1c3f-4889-a29b-5336b292b3cd None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:12.972801 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54477f89-abcc-4577-8c18-58fc651c52ff None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:12.972979 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b51c31-1c3f-4889-a29b-5336b292b3cd None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:12.973072 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54477f89-abcc-4577-8c18-58fc651c52ff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:12.973246 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b51c31-1c3f-4889-a29b-5336b292b3cd None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:12.981714 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b51c31-1c3f-4889-a29b-5336b292b3cd None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:12.983876 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54477f89-abcc-4577-8c18-58fc651c52ff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:12.984030 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b51c31-1c3f-4889-a29b-5336b292b3cd None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:12.985347 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54477f89-abcc-4577-8c18-58fc651c52ff None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:12.995454 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application [None req-d0b51c31-1c3f-4889-a29b-5336b292b3cd None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 05:58:12.995454 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:58:12.995454 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:58:12.995454 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:58:12.995454 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:58:12.995454 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:58:12.995454 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:58:12.995454 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:58:12.995454 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:58:12.995454 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:58:12.995454 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:58:12.995454 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:58:12.995454 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 05:58:12.995454 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 05:58:12.995454 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 05:58:12.995454 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 05:58:12.996615 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:58:12.996615 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:58:12.996615 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 05:58:12.996615 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 05:58:12.996615 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 05:58:12.996615 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 05:58:12.996615 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:58:12.996615 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:58:12.996615 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:58:12.996615 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:58:12.996615 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:58:12.996615 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application with Lock( Aug 24 05:58:12.996615 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:58:12.996615 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:58:12.996615 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:58:12.996615 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:58:12.996615 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:58:12.996615 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:58:12.997770 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:58:12.997770 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:58:12.997770 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 05:58:12.997770 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 05:58:12.997770 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 05:58:12.997770 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 05:58:12.997770 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 05:58:12.997770 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 05:58:12.997770 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 05:58:12.997770 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Aug 24 05:58:13.000181 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 427/852] 10.222.0.62 () {56 vars in 1117 bytes} [Wed Aug 24 05:58:12 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:58:13.000935 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-54477f89-abcc-4577-8c18-58fc651c52ff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:58:13.000935 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:58:13.000935 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:58:13.000935 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:58:13.000935 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:58:13.000935 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:58:13.000935 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:58:13.000935 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:58:13.000935 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:58:13.000935 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:58:13.000935 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:58:13.000935 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:58:13.000935 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 05:58:13.000935 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 05:58:13.000935 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 05:58:13.000935 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 05:58:13.002056 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:58:13.002056 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:58:13.002056 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:58:13.002056 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:58:13.002056 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:58:13.002056 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:58:13.002056 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:58:13.002056 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 24 05:58:13.002056 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:58:13.002056 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:58:13.002056 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:58:13.002056 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:58:13.002056 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:58:13.002056 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:58:13.002056 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:58:13.002056 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:58:13.002056 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 05:58:13.003134 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 05:58:13.003134 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:58:13.003134 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 24 05:58:13.003349 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b51c31-1c3f-4889-a29b-5336b292b3cd None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:13.009249 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 426/853] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Aug 24 05:58:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:58:13.009249 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54477f89-abcc-4577-8c18-58fc651c52ff None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:13.104457 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b51c31-1c3f-4889-a29b-5336b292b3cd None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:13.107649 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54477f89-abcc-4577-8c18-58fc651c52ff None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:13.137464 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41841e19-a7b7-4430-9b4b-118dd6721647 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:13.138067 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41841e19-a7b7-4430-9b4b-118dd6721647 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:13.138344 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41841e19-a7b7-4430-9b4b-118dd6721647 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:13.138615 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41841e19-a7b7-4430-9b4b-118dd6721647 None admin] PATH_INFO: `/v3/services` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:13.139774 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41841e19-a7b7-4430-9b4b-118dd6721647 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:58:13.140071 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41841e19-a7b7-4430-9b4b-118dd6721647 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:13.140659 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f70625-5158-4a44-99b3-906f3cb57acf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:13.141264 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f70625-5158-4a44-99b3-906f3cb57acf None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:13.141458 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41841e19-a7b7-4430-9b4b-118dd6721647 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:13.141549 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f70625-5158-4a44-99b3-906f3cb57acf None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:13.141809 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f70625-5158-4a44-99b3-906f3cb57acf None admin] PATH_INFO: `/v3/users` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:13.142526 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f70625-5158-4a44-99b3-906f3cb57acf None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:58:13.142831 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f70625-5158-4a44-99b3-906f3cb57acf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:13.144303 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f70625-5158-4a44-99b3-906f3cb57acf None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:13.145795 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 428/854] 10.222.0.62 () {56 vars in 1114 bytes} [Wed Aug 24 05:58:13 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:58:13.146072 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:13.146567 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41841e19-a7b7-4430-9b4b-118dd6721647 None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:13.171925 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 427/855] 10.222.0.62 () {56 vars in 1149 bytes} [Wed Aug 24 05:58:13 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:58:13.174069 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7489aa6d-7276-46ff-93ad-412300cefc11 None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:13.174533 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7489aa6d-7276-46ff-93ad-412300cefc11 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:13.175358 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7489aa6d-7276-46ff-93ad-412300cefc11 None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:13.177642 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 428/856] 10.222.0.62 () {54 vars in 927 bytes} [Wed Aug 24 05:58:13 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:58:13.179179 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7489aa6d-7276-46ff-93ad-412300cefc11 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:13.208545 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:13.266254 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41841e19-a7b7-4430-9b4b-118dd6721647 None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:13.304072 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7489aa6d-7276-46ff-93ad-412300cefc11 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:13.310594 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f55d16f-9f92-4589-8da2-77adeec4c9a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:13.310594 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f55d16f-9f92-4589-8da2-77adeec4c9a4 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:13.310851 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f55d16f-9f92-4589-8da2-77adeec4c9a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:13.311238 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f55d16f-9f92-4589-8da2-77adeec4c9a4 None admin] PATH_INFO: `/v3/domains` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:13.312410 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f55d16f-9f92-4589-8da2-77adeec4c9a4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:58:13.312838 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f55d16f-9f92-4589-8da2-77adeec4c9a4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:13.314678 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f55d16f-9f92-4589-8da2-77adeec4c9a4 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:13.328168 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 429/857] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Aug 24 05:58:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:58:13.329605 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a4deaf-359a-464e-a291-7cec9dbff4f7 None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:13.329987 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a4deaf-359a-464e-a291-7cec9dbff4f7 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:13.330354 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a4deaf-359a-464e-a291-7cec9dbff4f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:13.344985 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61a0e83-9708-4762-af17-11f45806c2d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:13.345596 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61a0e83-9708-4762-af17-11f45806c2d6 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:13.345874 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61a0e83-9708-4762-af17-11f45806c2d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:13.346143 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61a0e83-9708-4762-af17-11f45806c2d6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:13.350348 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-d6a4deaf-359a-464e-a291-7cec9dbff4f7 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:58:13.351501 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-d6a4deaf-359a-464e-a291-7cec9dbff4f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:58:13.416071 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:13.430328 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 430/858] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Aug 24 05:58:13 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 05:58:13.430328 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a4deaf-359a-464e-a291-7cec9dbff4f7 None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:13.430583 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61a0e83-9708-4762-af17-11f45806c2d6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:13.430583 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61a0e83-9708-4762-af17-11f45806c2d6 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:13.480176 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:13.539981 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 429/859] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Aug 24 05:58:13 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 05:58:13.545758 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61a0e83-9708-4762-af17-11f45806c2d6 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:13.557779 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a4deaf-359a-464e-a291-7cec9dbff4f7 None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:13.600392 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bc4e7d-42b1-4128-8bdf-b32e97453822 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:13.601108 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bc4e7d-42b1-4128-8bdf-b32e97453822 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:13.601395 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bc4e7d-42b1-4128-8bdf-b32e97453822 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:13.601669 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bc4e7d-42b1-4128-8bdf-b32e97453822 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:13.612406 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90bc4e7d-42b1-4128-8bdf-b32e97453822 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:13.614544 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90bc4e7d-42b1-4128-8bdf-b32e97453822 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:13.632718 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application [None req-90bc4e7d-42b1-4128-8bdf-b32e97453822 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 05:58:13.632718 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:58:13.632718 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:58:13.632718 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:58:13.632718 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:58:13.632718 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:58:13.632718 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:58:13.632718 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:58:13.632718 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:58:13.632718 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:58:13.632718 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:58:13.632718 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:58:13.632718 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 05:58:13.632718 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 05:58:13.632718 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 05:58:13.633648 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 05:58:13.633648 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:58:13.633648 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:58:13.633648 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:58:13.633648 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:58:13.633648 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:58:13.633648 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:58:13.633648 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:58:13.633648 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application with Lock( Aug 24 05:58:13.633648 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:58:13.633648 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:58:13.633648 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:58:13.633648 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:58:13.633648 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:58:13.633648 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:58:13.633648 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:58:13.633648 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:58:13.634540 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 05:58:13.634540 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 05:58:13.634540 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 05:58:13.634540 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 05:58:13.634540 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 05:58:13.634540 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 05:58:13.634540 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 05:58:13.634540 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Aug 24 05:58:13.635110 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 431/860] 10.222.0.62 () {56 vars in 1111 bytes} [Wed Aug 24 05:58:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:58:13.640369 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bc4e7d-42b1-4128-8bdf-b32e97453822 None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:13.651031 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:13.663555 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61a0e83-9708-4762-af17-11f45806c2d6 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:13.700345 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71614ecf-e06f-4ae3-a8b9-325971efc4cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:13.700953 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71614ecf-e06f-4ae3-a8b9-325971efc4cd None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:13.701231 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71614ecf-e06f-4ae3-a8b9-325971efc4cd None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:13.701568 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71614ecf-e06f-4ae3-a8b9-325971efc4cd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:13.712551 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71614ecf-e06f-4ae3-a8b9-325971efc4cd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:13.714012 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71614ecf-e06f-4ae3-a8b9-325971efc4cd None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:13.728834 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-71614ecf-e06f-4ae3-a8b9-325971efc4cd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:58:13.728834 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:58:13.728834 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:58:13.728834 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:58:13.728834 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:58:13.728834 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:58:13.728834 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:58:13.728834 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:58:13.728834 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:58:13.728834 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:58:13.728834 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:58:13.728834 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:58:13.728834 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 05:58:13.728834 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 05:58:13.728834 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 05:58:13.728834 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 05:58:13.729964 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:58:13.729964 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:58:13.729964 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:58:13.729964 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:58:13.729964 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:58:13.729964 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:58:13.729964 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:58:13.729964 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 24 05:58:13.729964 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:58:13.729964 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:58:13.729964 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:58:13.729964 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:58:13.729964 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:58:13.729964 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:58:13.729964 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:58:13.729964 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:58:13.729964 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 05:58:13.731126 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 05:58:13.731126 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:58:13.731126 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 24 05:58:13.735237 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 430/861] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Aug 24 05:58:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 05:58:13.736093 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71614ecf-e06f-4ae3-a8b9-325971efc4cd None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:13.758773 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:13.762863 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bc4e7d-42b1-4128-8bdf-b32e97453822 None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:13.801273 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ef16d7-50be-4f3f-bdd4-8bbf4219a6da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:13.801880 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ef16d7-50be-4f3f-bdd4-8bbf4219a6da None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:13.802163 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ef16d7-50be-4f3f-bdd4-8bbf4219a6da None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:13.802437 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ef16d7-50be-4f3f-bdd4-8bbf4219a6da None admin] PATH_INFO: `/v3/projects` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:13.803253 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ef16d7-50be-4f3f-bdd4-8bbf4219a6da None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:58:13.803671 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ef16d7-50be-4f3f-bdd4-8bbf4219a6da None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:13.805236 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ef16d7-50be-4f3f-bdd4-8bbf4219a6da None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:13.823104 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 432/862] 10.222.0.62 () {56 vars in 1148 bytes} [Wed Aug 24 05:58:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:58:13.823104 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ef16d7-50be-4f3f-bdd4-8bbf4219a6da None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:13.849627 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71614ecf-e06f-4ae3-a8b9-325971efc4cd None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:13.886624 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fbc9d8-b706-4384-8df8-1d41b14f5e6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:13.887252 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fbc9d8-b706-4384-8df8-1d41b14f5e6c None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:13.887536 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fbc9d8-b706-4384-8df8-1d41b14f5e6c None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:13.887919 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fbc9d8-b706-4384-8df8-1d41b14f5e6c None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:13.888526 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2fbc9d8-b706-4384-8df8-1d41b14f5e6c None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:13.890555 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2fbc9d8-b706-4384-8df8-1d41b14f5e6c None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:13.899339 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-c2fbc9d8-b706-4384-8df8-1d41b14f5e6c None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 05:58:13.899339 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:58:13.899339 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 05:58:13.899339 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 05:58:13.899339 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 05:58:13.899339 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 05:58:13.899339 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 05:58:13.899339 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 05:58:13.899339 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 05:58:13.899339 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 24 05:58:13.899339 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 05:58:13.899339 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 24 05:58:13.899339 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:58:13.899339 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:58:13.899339 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:58:13.899339 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:58:13.900295 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:58:13.900295 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:58:13.900295 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:58:13.900295 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:58:13.900295 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:58:13.900295 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:58:13.900295 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:58:13.900295 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 05:58:13.900295 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 05:58:13.900295 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 05:58:13.900295 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 05:58:13.900295 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:58:13.900295 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:58:13.900295 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:58:13.900295 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:58:13.900295 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:58:13.900295 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:58:13.901228 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:58:13.901228 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 24 05:58:13.901228 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:58:13.901228 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:58:13.901228 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:58:13.901228 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:58:13.901228 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:58:13.901228 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:58:13.901228 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:58:13.901228 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:58:13.901228 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 05:58:13.901228 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 05:58:13.901228 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 05:58:13.901228 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 24 05:58:13.905342 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 431/863] 10.222.0.62 () {56 vars in 1132 bytes} [Wed Aug 24 05:58:13 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:58:13.905999 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fbc9d8-b706-4384-8df8-1d41b14f5e6c None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:13.916898 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:13.930626 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ef16d7-50be-4f3f-bdd4-8bbf4219a6da None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:13.967837 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b65f4ba-c541-4042-99d0-b193b9778fac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:13.968801 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b65f4ba-c541-4042-99d0-b193b9778fac None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:13.969352 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b65f4ba-c541-4042-99d0-b193b9778fac None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:13.969635 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b65f4ba-c541-4042-99d0-b193b9778fac None admin] PATH_INFO: `/v3/domains` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:13.970416 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b65f4ba-c541-4042-99d0-b193b9778fac None admin] RBAC: Adding query filter params (name=Default) {{(pid=81851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:58:13.970728 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b65f4ba-c541-4042-99d0-b193b9778fac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:13.972777 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b65f4ba-c541-4042-99d0-b193b9778fac None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:13.989752 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 433/864] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Aug 24 05:58:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:58:13.989752 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b65f4ba-c541-4042-99d0-b193b9778fac None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:14.008908 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:14.015307 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fbc9d8-b706-4384-8df8-1d41b14f5e6c None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:14.052151 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6e1154-e4b4-4045-a476-32f5f425a069 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:14.052784 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6e1154-e4b4-4045-a476-32f5f425a069 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:14.053069 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6e1154-e4b4-4045-a476-32f5f425a069 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:14.053340 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6e1154-e4b4-4045-a476-32f5f425a069 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:14.054060 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6e1154-e4b4-4045-a476-32f5f425a069 None admin] RBAC: Adding query filter params (role.id=49e47cd157d743f6905f1db07ec878a7, scope.project.id=181f4762037e454489d9552cc603179c, user.id=14d4fed04fb144d08918bd5d4f808d23) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:58:14.054362 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6e1154-e4b4-4045-a476-32f5f425a069 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:14.055786 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6e1154-e4b4-4045-a476-32f5f425a069 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:14.062039 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 432/865] 10.222.0.62 () {56 vars in 1374 bytes} [Wed Aug 24 05:58:13 2022] GET /identity/v3/role_assignments?user.id=14d4fed04fb144d08918bd5d4f808d23&scope.project.id=181f4762037e454489d9552cc603179c&role.id=49e47cd157d743f6905f1db07ec878a7 => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:58:14.062712 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6e1154-e4b4-4045-a476-32f5f425a069 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:14.113716 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b65f4ba-c541-4042-99d0-b193b9778fac None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:14.161046 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69031b30-a6ef-4b19-9bbb-9c5bf8d089b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:14.161979 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:14.162099 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69031b30-a6ef-4b19-9bbb-9c5bf8d089b4 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:14.162284 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69031b30-a6ef-4b19-9bbb-9c5bf8d089b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:14.163211 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69031b30-a6ef-4b19-9bbb-9c5bf8d089b4 None admin] PATH_INFO: `/v3/services` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:14.163703 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69031b30-a6ef-4b19-9bbb-9c5bf8d089b4 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:58:14.164062 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69031b30-a6ef-4b19-9bbb-9c5bf8d089b4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:14.165493 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69031b30-a6ef-4b19-9bbb-9c5bf8d089b4 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:14.175634 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 434/866] 10.222.0.62 () {56 vars in 1126 bytes} [Wed Aug 24 05:58:13 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:58:14.177061 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69031b30-a6ef-4b19-9bbb-9c5bf8d089b4 None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:14.207999 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6e1154-e4b4-4045-a476-32f5f425a069 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:14.258534 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa5273f-335b-4a95-a388-34dfa4380f11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:14.259263 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa5273f-335b-4a95-a388-34dfa4380f11 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:14.259637 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa5273f-335b-4a95-a388-34dfa4380f11 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:14.259916 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa5273f-335b-4a95-a388-34dfa4380f11 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:14.269757 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa5273f-335b-4a95-a388-34dfa4380f11 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:14.271210 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa5273f-335b-4a95-a388-34dfa4380f11 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:14.284942 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-2fa5273f-335b-4a95-a388-34dfa4380f11 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 05:58:14.284942 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:58:14.284942 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:58:14.284942 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:58:14.284942 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:58:14.284942 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:58:14.284942 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:58:14.284942 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:58:14.284942 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:58:14.284942 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:58:14.284942 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:58:14.284942 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:58:14.284942 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 05:58:14.284942 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 05:58:14.284942 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 05:58:14.285853 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 05:58:14.285853 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:58:14.285853 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:58:14.285853 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:58:14.285853 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:58:14.285853 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:58:14.285853 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:58:14.285853 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:58:14.285853 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 24 05:58:14.285853 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:58:14.285853 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:58:14.285853 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:58:14.285853 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:58:14.285853 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:58:14.285853 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:58:14.285853 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:58:14.285853 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:58:14.286743 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 05:58:14.286743 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 05:58:14.286743 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 05:58:14.286743 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 05:58:14.286743 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 05:58:14.286743 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 05:58:14.286743 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 05:58:14.286743 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 24 05:58:14.288350 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 433/867] 10.222.0.62 () {56 vars in 1111 bytes} [Wed Aug 24 05:58:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:58:14.294469 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa5273f-335b-4a95-a388-34dfa4380f11 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:14.296263 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:14.321295 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69031b30-a6ef-4b19-9bbb-9c5bf8d089b4 None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:14.366345 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7d3cbd-3e7b-4c75-9df1-3462474d93be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:14.367291 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7d3cbd-3e7b-4c75-9df1-3462474d93be None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:14.367842 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7d3cbd-3e7b-4c75-9df1-3462474d93be None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:14.368216 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7d3cbd-3e7b-4c75-9df1-3462474d93be None admin] PATH_INFO: `/v3/services` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:14.369127 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7d3cbd-3e7b-4c75-9df1-3462474d93be None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:58:14.370317 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7d3cbd-3e7b-4c75-9df1-3462474d93be None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:14.371797 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7d3cbd-3e7b-4c75-9df1-3462474d93be None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:14.378329 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 435/868] 10.222.0.62 () {56 vars in 1126 bytes} [Wed Aug 24 05:58:14 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:58:14.406594 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa5273f-335b-4a95-a388-34dfa4380f11 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:14.440216 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:14.445788 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b703eb-216b-4d39-ad00-3106ca132654 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:14.446831 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b703eb-216b-4d39-ad00-3106ca132654 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:14.447107 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b703eb-216b-4d39-ad00-3106ca132654 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:14.447372 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b703eb-216b-4d39-ad00-3106ca132654 None admin] PATH_INFO: `/v3/projects` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:14.448134 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b703eb-216b-4d39-ad00-3106ca132654 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:58:14.448439 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b703eb-216b-4d39-ad00-3106ca132654 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:14.449990 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b703eb-216b-4d39-ad00-3106ca132654 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:14.464732 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 434/869] 10.222.0.62 () {56 vars in 1148 bytes} [Wed Aug 24 05:58:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:58:14.471312 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7d3cbd-3e7b-4c75-9df1-3462474d93be None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:14.588865 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7d3cbd-3e7b-4c75-9df1-3462474d93be None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:14.625520 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c96317-a816-4390-9bc7-4b36d7cff756 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:14.627124 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c96317-a816-4390-9bc7-4b36d7cff756 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:14.627613 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c96317-a816-4390-9bc7-4b36d7cff756 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:14.628089 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c96317-a816-4390-9bc7-4b36d7cff756 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:14.629054 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c96317-a816-4390-9bc7-4b36d7cff756 None admin] RBAC: Adding query filter params (role.id=5d95a5fc0f324a92905533aa8d34ac15, scope.project.id=181f4762037e454489d9552cc603179c, user.id=b82726a12436435cb931b986ef884c2e) {{(pid=81851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:58:14.629586 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c96317-a816-4390-9bc7-4b36d7cff756 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:14.631221 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c96317-a816-4390-9bc7-4b36d7cff756 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:14.637984 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 436/870] 10.222.0.62 () {56 vars in 1374 bytes} [Wed Aug 24 05:58:14 2022] GET /identity/v3/role_assignments?user.id=b82726a12436435cb931b986ef884c2e&scope.project.id=181f4762037e454489d9552cc603179c&role.id=5d95a5fc0f324a92905533aa8d34ac15 => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:58:15.084660 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9f15a3-7a5b-4fbc-810b-81be7747f19e None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:15.085068 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9f15a3-7a5b-4fbc-810b-81be7747f19e None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:15.085260 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9f15a3-7a5b-4fbc-810b-81be7747f19e None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:15.086679 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 435/871] 10.222.0.62 () {54 vars in 927 bytes} [Wed Aug 24 05:58:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:58:15.092024 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a721def-3a89-46c3-85d9-1c5f065b1adc None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:15.092552 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a721def-3a89-46c3-85d9-1c5f065b1adc None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:15.093046 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a721def-3a89-46c3-85d9-1c5f065b1adc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:15.111378 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-7a721def-3a89-46c3-85d9-1c5f065b1adc None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:58:15.112936 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-7a721def-3a89-46c3-85d9-1c5f065b1adc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:58:15.116199 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:15.184953 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 437/872] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Aug 24 05:58:15 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 05:58:15.454486 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9f15a3-7a5b-4fbc-810b-81be7747f19e None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:15.562808 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9f15a3-7a5b-4fbc-810b-81be7747f19e None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:15.599034 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba80e8f3-31cd-42e0-ba11-ae2ec0f54162 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:15.599645 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba80e8f3-31cd-42e0-ba11-ae2ec0f54162 None admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:15.599925 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba80e8f3-31cd-42e0-ba11-ae2ec0f54162 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:15.600191 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba80e8f3-31cd-42e0-ba11-ae2ec0f54162 None admin] PATH_INFO: `/v3/services` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:15.600868 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba80e8f3-31cd-42e0-ba11-ae2ec0f54162 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:15.602228 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba80e8f3-31cd-42e0-ba11-ae2ec0f54162 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:15.697130 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 436/873] 10.222.0.62 () {60 vars in 1141 bytes} [Wed Aug 24 05:58:15 2022] POST /identity/v3/services => generated 251 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:58:16.122120 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a58f37-23d5-47f5-8cfc-36bf7460c1c1 None None] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:16.123196 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a58f37-23d5-47f5-8cfc-36bf7460c1c1 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:16.123689 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a58f37-23d5-47f5-8cfc-36bf7460c1c1 None None] PATH_INFO: `/` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:16.125630 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 438/874] 10.222.0.62 () {54 vars in 927 bytes} [Wed Aug 24 05:58:16 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:58:16.132950 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207d2b3b-a2e1-4aa8-970b-09623baf2329 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:16.133232 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207d2b3b-a2e1-4aa8-970b-09623baf2329 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:16.133503 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207d2b3b-a2e1-4aa8-970b-09623baf2329 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:16.152157 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-207d2b3b-a2e1-4aa8-970b-09623baf2329 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:58:16.153361 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-207d2b3b-a2e1-4aa8-970b-09623baf2329 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:58:16.203652 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:16.230377 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 437/875] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Aug 24 05:58:16 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 05:58:16.376704 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf13259-b78a-49e6-b0e4-2c1dd80ed6fd None None] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:16.376895 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf13259-b78a-49e6-b0e4-2c1dd80ed6fd None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:16.377173 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf13259-b78a-49e6-b0e4-2c1dd80ed6fd None None] PATH_INFO: `/` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:16.378763 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 439/876] 10.222.0.62 () {54 vars in 927 bytes} [Wed Aug 24 05:58:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:58:16.384003 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b10fe4-95b0-4c32-84f0-2eef857e8ed5 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:16.384295 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b10fe4-95b0-4c32-84f0-2eef857e8ed5 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:16.384594 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b10fe4-95b0-4c32-84f0-2eef857e8ed5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:16.403125 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-e1b10fe4-95b0-4c32-84f0-2eef857e8ed5 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:58:16.404307 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-e1b10fe4-95b0-4c32-84f0-2eef857e8ed5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:58:16.473305 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 438/877] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Aug 24 05:58:16 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 05:58:16.502213 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf13259-b78a-49e6-b0e4-2c1dd80ed6fd None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:16.603455 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf13259-b78a-49e6-b0e4-2c1dd80ed6fd None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:16.637694 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a251590-b775-4a77-9e36-0d4440efa075 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:16.638240 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a251590-b775-4a77-9e36-0d4440efa075 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:16.638516 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a251590-b775-4a77-9e36-0d4440efa075 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:16.638811 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a251590-b775-4a77-9e36-0d4440efa075 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:16.639387 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a251590-b775-4a77-9e36-0d4440efa075 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:16.640765 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a251590-b775-4a77-9e36-0d4440efa075 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:16.649330 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application [None req-6a251590-b775-4a77-9e36-0d4440efa075 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 05:58:16.649330 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:58:16.649330 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 05:58:16.649330 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 05:58:16.649330 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 05:58:16.649330 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 05:58:16.649330 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 05:58:16.649330 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 05:58:16.649330 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 05:58:16.649330 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Aug 24 05:58:16.649330 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 05:58:16.649330 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Aug 24 05:58:16.649330 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:58:16.649330 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:58:16.649330 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:58:16.649330 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:58:16.649330 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:58:16.650301 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:58:16.650301 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:58:16.650301 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:58:16.650301 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:58:16.650301 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:58:16.650301 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:58:16.650301 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 05:58:16.650301 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 05:58:16.650301 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 05:58:16.650301 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 05:58:16.650301 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:58:16.650301 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:58:16.650301 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:58:16.650301 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:58:16.650301 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:58:16.650301 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:58:16.651127 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:58:16.651127 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application with Lock( Aug 24 05:58:16.651127 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:58:16.651127 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:58:16.651127 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:58:16.651127 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:58:16.651127 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:58:16.651127 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:58:16.651127 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:58:16.651127 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:58:16.651127 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 05:58:16.651127 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 05:58:16.651127 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 05:58:16.651127 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Aug 24 05:58:16.655739 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 440/878] 10.222.0.62 () {56 vars in 1126 bytes} [Wed Aug 24 05:58:16 2022] GET /identity/v3/services/object-store => generated 93 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:58:16.657576 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b10fe4-95b0-4c32-84f0-2eef857e8ed5 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:16.709562 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cd1d32-d0ac-4c07-a45a-bbf79ba36974 None None] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:16.710259 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cd1d32-d0ac-4c07-a45a-bbf79ba36974 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:16.710259 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cd1d32-d0ac-4c07-a45a-bbf79ba36974 None None] PATH_INFO: `/` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:16.711700 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 441/879] 10.222.0.62 () {54 vars in 927 bytes} [Wed Aug 24 05:58:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 05:58:16.717578 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f798576-dfa8-4b85-8dea-193fd26a161a None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:16.717931 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f798576-dfa8-4b85-8dea-193fd26a161a None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:16.718265 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f798576-dfa8-4b85-8dea-193fd26a161a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:16.726978 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:16.738664 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-0f798576-dfa8-4b85-8dea-193fd26a161a None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:58:16.740200 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-0f798576-dfa8-4b85-8dea-193fd26a161a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:58:16.767323 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b10fe4-95b0-4c32-84f0-2eef857e8ed5 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:16.779046 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:16.809201 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4557bf7b-ec05-4c61-909c-b79635663b3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:16.809808 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4557bf7b-ec05-4c61-909c-b79635663b3e None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:16.810084 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4557bf7b-ec05-4c61-909c-b79635663b3e None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:16.810349 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4557bf7b-ec05-4c61-909c-b79635663b3e None admin] PATH_INFO: `/v3/services` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:16.810997 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4557bf7b-ec05-4c61-909c-b79635663b3e None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:58:16.811290 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4557bf7b-ec05-4c61-909c-b79635663b3e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:16.812684 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4557bf7b-ec05-4c61-909c-b79635663b3e None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:16.818565 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 439/880] 10.222.0.62 () {56 vars in 1122 bytes} [Wed Aug 24 05:58:16 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:58:16.821357 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4557bf7b-ec05-4c61-909c-b79635663b3e None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:16.821753 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 442/881] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Aug 24 05:58:16 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 05:58:16.824231 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f798576-dfa8-4b85-8dea-193fd26a161a None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:16.938903 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f798576-dfa8-4b85-8dea-193fd26a161a None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:16.960604 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4557bf7b-ec05-4c61-909c-b79635663b3e None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:16.981281 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1794c53c-baab-4cc0-9575-c1ab2f2517a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:16.981891 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1794c53c-baab-4cc0-9575-c1ab2f2517a9 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:16.982168 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1794c53c-baab-4cc0-9575-c1ab2f2517a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:16.982939 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1794c53c-baab-4cc0-9575-c1ab2f2517a9 None admin] PATH_INFO: `/v3/services` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:16.983845 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1794c53c-baab-4cc0-9575-c1ab2f2517a9 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:58:16.984149 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1794c53c-baab-4cc0-9575-c1ab2f2517a9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:16.986206 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1794c53c-baab-4cc0-9575-c1ab2f2517a9 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:16.987648 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:16.996665 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 443/882] 10.222.0.62 () {56 vars in 1122 bytes} [Wed Aug 24 05:58:16 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:58:17.005223 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4fadb0-5107-4fc2-a74f-c1da4536578c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:17.005852 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4fadb0-5107-4fc2-a74f-c1da4536578c None admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:17.006124 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4fadb0-5107-4fc2-a74f-c1da4536578c None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:17.006391 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4fadb0-5107-4fc2-a74f-c1da4536578c None admin] PATH_INFO: `/v3/services` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:17.007049 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4fadb0-5107-4fc2-a74f-c1da4536578c None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:17.008432 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4fadb0-5107-4fc2-a74f-c1da4536578c None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:17.096411 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1794c53c-baab-4cc0-9575-c1ab2f2517a9 None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:17.103921 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 440/883] 10.222.0.62 () {60 vars in 1141 bytes} [Wed Aug 24 05:58:16 2022] POST /identity/v3/services => generated 269 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:58:17.218547 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1794c53c-baab-4cc0-9575-c1ab2f2517a9 None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:17.263199 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50df0f5-391d-4017-b6da-eea9ea073c2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:17.264259 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50df0f5-391d-4017-b6da-eea9ea073c2e None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:17.264744 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50df0f5-391d-4017-b6da-eea9ea073c2e None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:17.265218 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50df0f5-391d-4017-b6da-eea9ea073c2e None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:17.277809 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50df0f5-391d-4017-b6da-eea9ea073c2e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:58:17.283658 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application [None req-b50df0f5-391d-4017-b6da-eea9ea073c2e None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 05:58:17.283658 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:58:17.283658 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:58:17.283658 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:58:17.283658 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:58:17.283658 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:58:17.283658 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:58:17.283658 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:58:17.283658 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:58:17.283658 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:58:17.283658 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:58:17.283658 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:58:17.283658 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:58:17.283658 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:58:17.283658 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:58:17.283658 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:58:17.284626 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:58:17.284626 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:58:17.284626 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:58:17.284626 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:58:17.284626 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:58:17.284626 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:58:17.284626 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:58:17.284626 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:58:17.284626 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:58:17.284626 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:58:17.284626 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:58:17.284626 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application with Lock( Aug 24 05:58:17.284626 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:58:17.284626 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:58:17.284626 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:58:17.284626 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:58:17.285485 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:58:17.285485 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:58:17.285485 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:58:17.285485 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:58:17.285485 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:58:17.285485 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:58:17.285485 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:58:17.285485 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:58:17.285485 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:58:17.285485 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:58:17.285485 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 05:58:17.285485 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Aug 24 05:58:17.285485 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 444/884] 10.222.0.62 () {56 vars in 1120 bytes} [Wed Aug 24 05:58:17 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 05:58:17.292721 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4fadb0-5107-4fc2-a74f-c1da4536578c None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:17.328405 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:17.401200 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4fadb0-5107-4fc2-a74f-c1da4536578c None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:17.436257 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cea83fa-91f0-4576-a453-7d3e53ad3dca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:17.436902 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cea83fa-91f0-4576-a453-7d3e53ad3dca None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:17.437180 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cea83fa-91f0-4576-a453-7d3e53ad3dca None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:17.437450 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cea83fa-91f0-4576-a453-7d3e53ad3dca None admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:17.438105 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cea83fa-91f0-4576-a453-7d3e53ad3dca None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:58:17.438403 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cea83fa-91f0-4576-a453-7d3e53ad3dca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:17.439767 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cea83fa-91f0-4576-a453-7d3e53ad3dca None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:17.448390 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 441/885] 10.222.0.62 () {56 vars in 1115 bytes} [Wed Aug 24 05:58:17 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:58:17.454778 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50df0f5-391d-4017-b6da-eea9ea073c2e None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:17.520549 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:17.569837 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50df0f5-391d-4017-b6da-eea9ea073c2e None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:17.605112 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6f6239-5703-4dbd-b4e9-ce99341c750e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:17.605752 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6f6239-5703-4dbd-b4e9-ce99341c750e None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:17.606034 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6f6239-5703-4dbd-b4e9-ce99341c750e None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:17.606308 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6f6239-5703-4dbd-b4e9-ce99341c750e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:17.618092 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6f6239-5703-4dbd-b4e9-ce99341c750e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:17.619571 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6f6239-5703-4dbd-b4e9-ce99341c750e None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:17.634661 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application [None req-4c6f6239-5703-4dbd-b4e9-ce99341c750e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:58:17.634661 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:58:17.634661 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:58:17.634661 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:58:17.634661 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:58:17.634661 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:58:17.634661 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:58:17.634661 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:58:17.634661 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:58:17.634661 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:58:17.634661 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:58:17.634661 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:58:17.634661 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 05:58:17.634661 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 05:58:17.634661 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 05:58:17.634661 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 05:58:17.635650 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:58:17.635650 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:58:17.635650 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:58:17.635650 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:58:17.635650 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:58:17.635650 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:58:17.635650 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:58:17.635650 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application with Lock( Aug 24 05:58:17.635650 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:58:17.635650 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:58:17.635650 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:58:17.635650 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:58:17.635650 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:58:17.635650 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:58:17.635650 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:58:17.635650 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:58:17.635650 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 05:58:17.636875 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 05:58:17.636875 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:58:17.636875 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Aug 24 05:58:17.636875 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 445/886] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Aug 24 05:58:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:58:17.645493 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cea83fa-91f0-4576-a453-7d3e53ad3dca None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:17.695395 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bdff57-8cbb-4f2e-a1b4-62d6c109c38e None None] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:17.695675 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bdff57-8cbb-4f2e-a1b4-62d6c109c38e None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:17.695946 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bdff57-8cbb-4f2e-a1b4-62d6c109c38e None None] PATH_INFO: `/` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:17.697627 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 446/887] 10.222.0.62 () {54 vars in 927 bytes} [Wed Aug 24 05:58:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 05:58:17.705416 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebe04ce-f422-434b-b93e-3d283c4b732b None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:17.705416 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebe04ce-f422-434b-b93e-3d283c4b732b None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:17.705416 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebe04ce-f422-434b-b93e-3d283c4b732b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:17.721750 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-0ebe04ce-f422-434b-b93e-3d283c4b732b None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:58:17.723103 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-0ebe04ce-f422-434b-b93e-3d283c4b732b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:58:17.756972 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:17.765033 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cea83fa-91f0-4576-a453-7d3e53ad3dca None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:17.802468 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a47aab1-a2b5-4e0c-acd5-b825274d3de4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:17.803454 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a47aab1-a2b5-4e0c-acd5-b825274d3de4 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:17.803814 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a47aab1-a2b5-4e0c-acd5-b825274d3de4 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:17.804193 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a47aab1-a2b5-4e0c-acd5-b825274d3de4 None admin] PATH_INFO: `/v3/domains` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:17.804998 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a47aab1-a2b5-4e0c-acd5-b825274d3de4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:58:17.805376 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a47aab1-a2b5-4e0c-acd5-b825274d3de4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:17.812518 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a47aab1-a2b5-4e0c-acd5-b825274d3de4 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:17.818458 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 447/888] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Aug 24 05:58:17 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 05:58:17.824538 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 442/889] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Aug 24 05:58:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:58:17.830420 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebe04ce-f422-434b-b93e-3d283c4b732b None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:17.866287 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:17.937032 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebe04ce-f422-434b-b93e-3d283c4b732b None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:17.975973 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1aacd3-b021-46a9-8d11-853ad9978ad6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:17.976630 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1aacd3-b021-46a9-8d11-853ad9978ad6 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:17.976910 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1aacd3-b021-46a9-8d11-853ad9978ad6 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:17.977183 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1aacd3-b021-46a9-8d11-853ad9978ad6 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:17.984825 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1aacd3-b021-46a9-8d11-853ad9978ad6 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:17.986261 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1aacd3-b021-46a9-8d11-853ad9978ad6 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:17.997577 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application [None req-0c1aacd3-b021-46a9-8d11-853ad9978ad6 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 05:58:17.997577 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:58:17.997577 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:58:17.997577 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:58:17.997577 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:58:17.997577 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:58:17.997577 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:58:17.997577 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:58:17.997577 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:58:17.997577 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:58:17.997577 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:58:17.997577 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:58:17.997577 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 05:58:17.997577 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 05:58:17.997577 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 05:58:17.997577 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 05:58:17.998552 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:58:17.998552 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:58:17.998552 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 05:58:17.998552 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 05:58:17.998552 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 05:58:17.998552 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 05:58:17.998552 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:58:17.998552 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:58:17.998552 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:58:17.998552 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:58:17.998552 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:58:17.998552 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application with Lock( Aug 24 05:58:17.998552 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:58:17.998552 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:58:17.998552 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:58:17.998552 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:58:17.998552 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:58:17.998552 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:58:17.999498 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:58:17.999498 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:58:17.999498 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 05:58:17.999498 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 05:58:17.999498 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 05:58:17.999498 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 05:58:17.999498 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 05:58:17.999498 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 05:58:17.999498 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 05:58:17.999498 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Aug 24 05:58:17.999498 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 448/890] 10.222.0.62 () {56 vars in 1117 bytes} [Wed Aug 24 05:58:17 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:58:18.006332 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a47aab1-a2b5-4e0c-acd5-b825274d3de4 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:18.109967 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a47aab1-a2b5-4e0c-acd5-b825274d3de4 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:18.133761 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1aacd3-b021-46a9-8d11-853ad9978ad6 None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:18.151180 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb59dcd2-f50a-4328-9456-cc76e5ffd689 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:18.151823 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb59dcd2-f50a-4328-9456-cc76e5ffd689 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:18.152103 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb59dcd2-f50a-4328-9456-cc76e5ffd689 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:18.153139 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb59dcd2-f50a-4328-9456-cc76e5ffd689 None admin] PATH_INFO: `/v3/users` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:18.153350 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb59dcd2-f50a-4328-9456-cc76e5ffd689 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:58:18.153620 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb59dcd2-f50a-4328-9456-cc76e5ffd689 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:18.155901 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb59dcd2-f50a-4328-9456-cc76e5ffd689 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:18.179506 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 443/891] 10.222.0.62 () {56 vars in 1149 bytes} [Wed Aug 24 05:58:18 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:58:18.185121 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb59dcd2-f50a-4328-9456-cc76e5ffd689 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:18.204621 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:18.256153 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:18.262370 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1aacd3-b021-46a9-8d11-853ad9978ad6 None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:18.300579 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d96636-d3b8-4829-90d8-4cfb98ac52d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:18.302130 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d96636-d3b8-4829-90d8-4cfb98ac52d1 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:18.302354 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d96636-d3b8-4829-90d8-4cfb98ac52d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:18.302631 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d96636-d3b8-4829-90d8-4cfb98ac52d1 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:18.303757 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d96636-d3b8-4829-90d8-4cfb98ac52d1 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:18.305885 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d96636-d3b8-4829-90d8-4cfb98ac52d1 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:18.316246 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application [None req-d6d96636-d3b8-4829-90d8-4cfb98ac52d1 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 05:58:18.316246 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:58:18.316246 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 05:58:18.316246 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 05:58:18.316246 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 05:58:18.316246 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 05:58:18.316246 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 05:58:18.316246 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 05:58:18.316246 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 05:58:18.316246 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Aug 24 05:58:18.316246 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 05:58:18.316246 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Aug 24 05:58:18.316246 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:58:18.316246 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:58:18.316246 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:58:18.316246 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:58:18.316246 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:58:18.317299 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:58:18.317299 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:58:18.317299 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:58:18.317299 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:58:18.317299 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:58:18.317299 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:58:18.317299 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 05:58:18.317299 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 05:58:18.317299 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 05:58:18.317299 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 05:58:18.317299 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:58:18.317299 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:58:18.317299 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:58:18.317299 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:58:18.317299 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:58:18.317299 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:58:18.318142 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:58:18.318142 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application with Lock( Aug 24 05:58:18.318142 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:58:18.318142 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:58:18.318142 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:58:18.318142 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:58:18.318142 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:58:18.318142 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:58:18.318142 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:58:18.318142 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:58:18.318142 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 05:58:18.318142 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 05:58:18.318142 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 05:58:18.318142 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Aug 24 05:58:18.318142 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 449/892] 10.222.0.62 () {56 vars in 1114 bytes} [Wed Aug 24 05:58:18 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 05:58:18.324562 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d96636-d3b8-4829-90d8-4cfb98ac52d1 None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:18.334722 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb59dcd2-f50a-4328-9456-cc76e5ffd689 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:18.387564 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b1c034-075a-4c6f-be36-6ea526bb9b47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:18.389622 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b1c034-075a-4c6f-be36-6ea526bb9b47 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:18.390363 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b1c034-075a-4c6f-be36-6ea526bb9b47 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:18.390843 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b1c034-075a-4c6f-be36-6ea526bb9b47 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:18.438107 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d96636-d3b8-4829-90d8-4cfb98ac52d1 None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:18.473545 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71c9981-637f-43f7-a1c9-bcf9c2951ce5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:18.474118 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71c9981-637f-43f7-a1c9-bcf9c2951ce5 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:18.474392 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71c9981-637f-43f7-a1c9-bcf9c2951ce5 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:18.474656 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71c9981-637f-43f7-a1c9-bcf9c2951ce5 None admin] PATH_INFO: `/v3/services` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:18.475528 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71c9981-637f-43f7-a1c9-bcf9c2951ce5 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:58:18.475640 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71c9981-637f-43f7-a1c9-bcf9c2951ce5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:18.477012 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71c9981-637f-43f7-a1c9-bcf9c2951ce5 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:18.487019 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 450/893] 10.222.0.62 () {56 vars in 1114 bytes} [Wed Aug 24 05:58:18 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:58:18.488071 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71c9981-637f-43f7-a1c9-bcf9c2951ce5 None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:18.489730 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b1c034-075a-4c6f-be36-6ea526bb9b47 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:18.491282 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b1c034-075a-4c6f-be36-6ea526bb9b47 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:18.506746 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:18.522650 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:18.599240 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 444/894] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Aug 24 05:58:18 2022] GET /identity/v3/auth/tokens => generated 1372 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 05:58:18.604624 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b1c034-075a-4c6f-be36-6ea526bb9b47 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:18.629300 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71c9981-637f-43f7-a1c9-bcf9c2951ce5 None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:18.665355 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdac3f0-5c5e-4385-8364-bac2e20b1974 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:18.666297 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdac3f0-5c5e-4385-8364-bac2e20b1974 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:18.666799 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdac3f0-5c5e-4385-8364-bac2e20b1974 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:18.667264 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdac3f0-5c5e-4385-8364-bac2e20b1974 None admin] PATH_INFO: `/v3/services` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:18.668081 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fdac3f0-5c5e-4385-8364-bac2e20b1974 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:58:18.668630 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fdac3f0-5c5e-4385-8364-bac2e20b1974 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:18.671143 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fdac3f0-5c5e-4385-8364-bac2e20b1974 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:18.677150 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 451/895] 10.222.0.62 () {56 vars in 1114 bytes} [Wed Aug 24 05:58:18 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:58:18.682869 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdac3f0-5c5e-4385-8364-bac2e20b1974 None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:18.756877 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b1c034-075a-4c6f-be36-6ea526bb9b47 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:18.771401 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:18.793685 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f93f112-af78-4453-a88f-39f690640d24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:18.794622 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f93f112-af78-4453-a88f-39f690640d24 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:18.795923 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f93f112-af78-4453-a88f-39f690640d24 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:18.796422 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f93f112-af78-4453-a88f-39f690640d24 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:18.802818 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:18.807890 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f93f112-af78-4453-a88f-39f690640d24 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:18.809771 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f93f112-af78-4453-a88f-39f690640d24 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:18.819661 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdac3f0-5c5e-4385-8364-bac2e20b1974 None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:18.828079 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-3f93f112-af78-4453-a88f-39f690640d24 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:58:18.828079 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:58:18.828079 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:58:18.828079 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:58:18.828079 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:58:18.828079 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:58:18.828079 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:58:18.828079 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:58:18.828079 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:58:18.828079 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:58:18.828079 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:58:18.828079 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:58:18.828079 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 05:58:18.828079 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 05:58:18.828079 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 05:58:18.828079 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 05:58:18.829311 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:58:18.829311 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:58:18.829311 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:58:18.829311 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:58:18.829311 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:58:18.829311 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:58:18.829311 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:58:18.829311 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 24 05:58:18.829311 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:58:18.829311 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:58:18.829311 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:58:18.829311 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:58:18.829311 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:58:18.829311 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:58:18.829311 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:58:18.829311 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:58:18.829311 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 05:58:18.830212 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 05:58:18.830212 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:58:18.830212 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 24 05:58:18.834809 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 445/896] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Aug 24 05:58:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:58:18.835704 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f93f112-af78-4453-a88f-39f690640d24 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:18.864733 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910d5c7b-7626-40e5-92ac-63bca79916f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:18.865822 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910d5c7b-7626-40e5-92ac-63bca79916f6 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:18.866295 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910d5c7b-7626-40e5-92ac-63bca79916f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:18.866762 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910d5c7b-7626-40e5-92ac-63bca79916f6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:18.867653 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910d5c7b-7626-40e5-92ac-63bca79916f6 None admin] RBAC: Adding query filter params (interface=public, service_id=10a4312cff4f44bd9cab4620dd3ac974, region_id=RegionOne) {{(pid=81851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:58:18.868217 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910d5c7b-7626-40e5-92ac-63bca79916f6 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:18.869998 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910d5c7b-7626-40e5-92ac-63bca79916f6 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:18.876436 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 452/897] 10.222.0.62 () {56 vars in 1251 bytes} [Wed Aug 24 05:58:18 2022] GET /identity/v3/endpoints?service_id=10a4312cff4f44bd9cab4620dd3ac974&interface=public®ion_id=RegionOne => generated 194 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:58:18.881986 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910d5c7b-7626-40e5-92ac-63bca79916f6 None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:18.966869 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f93f112-af78-4453-a88f-39f690640d24 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:19.004793 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910d5c7b-7626-40e5-92ac-63bca79916f6 None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:19.007362 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1a6dc8-472b-4764-9f3a-4969b3fbd8ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:19.007971 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1a6dc8-472b-4764-9f3a-4969b3fbd8ef None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:19.008256 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1a6dc8-472b-4764-9f3a-4969b3fbd8ef None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:19.008553 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1a6dc8-472b-4764-9f3a-4969b3fbd8ef None admin] PATH_INFO: `/v3/domains` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:19.009198 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1a6dc8-472b-4764-9f3a-4969b3fbd8ef None admin] RBAC: Adding query filter params (name=Default) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:58:19.009502 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1a6dc8-472b-4764-9f3a-4969b3fbd8ef None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:19.010873 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1a6dc8-472b-4764-9f3a-4969b3fbd8ef None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:19.024474 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 446/898] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Aug 24 05:58:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:58:19.026060 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ede4952-357d-4fa7-b74c-db26f248b703 None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:19.026432 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ede4952-357d-4fa7-b74c-db26f248b703 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:19.026824 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ede4952-357d-4fa7-b74c-db26f248b703 None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:19.028295 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 447/899] 10.222.0.62 () {54 vars in 927 bytes} [Wed Aug 24 05:58:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:58:19.033124 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ede4952-357d-4fa7-b74c-db26f248b703 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:19.044815 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e1c3f2-6ef7-4d7b-83e8-3209a6f4ace8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:19.046289 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e1c3f2-6ef7-4d7b-83e8-3209a6f4ace8 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:19.046787 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e1c3f2-6ef7-4d7b-83e8-3209a6f4ace8 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:19.047244 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e1c3f2-6ef7-4d7b-83e8-3209a6f4ace8 None admin] PATH_INFO: `/v3/services` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:19.048137 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e1c3f2-6ef7-4d7b-83e8-3209a6f4ace8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:19.049825 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e1c3f2-6ef7-4d7b-83e8-3209a6f4ace8 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:19.056224 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 453/900] 10.222.0.62 () {56 vars in 1087 bytes} [Wed Aug 24 05:58:18 2022] GET /identity/v3/services => generated 1268 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 05:58:19.057702 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0daa184-fa88-4a82-8d2b-98e6533a0801 None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:19.057993 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0daa184-fa88-4a82-8d2b-98e6533a0801 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:19.058270 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0daa184-fa88-4a82-8d2b-98e6533a0801 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:19.062502 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:19.086718 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-e0daa184-fa88-4a82-8d2b-98e6533a0801 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:58:19.088207 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-e0daa184-fa88-4a82-8d2b-98e6533a0801 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:58:19.103748 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:19.162451 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ede4952-357d-4fa7-b74c-db26f248b703 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:19.199137 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 454/901] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Aug 24 05:58:19 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 05:58:19.199137 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a9ab89-7180-47f3-a25d-6ed3c0acca45 None None] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:19.199137 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a9ab89-7180-47f3-a25d-6ed3c0acca45 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:19.199137 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a9ab89-7180-47f3-a25d-6ed3c0acca45 None None] PATH_INFO: `/` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:19.200644 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 455/902] 10.222.0.62 () {54 vars in 927 bytes} [Wed Aug 24 05:58:19 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:58:19.204017 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5437a562-7303-4a76-a6bc-ac939e51331e None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:19.204303 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5437a562-7303-4a76-a6bc-ac939e51331e None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:19.204615 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5437a562-7303-4a76-a6bc-ac939e51331e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:19.208850 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477c71a9-49a7-4752-be88-e04f24b5657b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:19.209571 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477c71a9-49a7-4752-be88-e04f24b5657b None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:19.209926 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477c71a9-49a7-4752-be88-e04f24b5657b None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:19.210271 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477c71a9-49a7-4752-be88-e04f24b5657b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:19.220167 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477c71a9-49a7-4752-be88-e04f24b5657b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:19.221603 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477c71a9-49a7-4752-be88-e04f24b5657b None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:19.224402 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-5437a562-7303-4a76-a6bc-ac939e51331e None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:58:19.225630 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-5437a562-7303-4a76-a6bc-ac939e51331e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:58:19.233477 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-477c71a9-49a7-4752-be88-e04f24b5657b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 05:58:19.233477 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:58:19.233477 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:58:19.233477 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:58:19.233477 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:58:19.233477 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:58:19.233477 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:58:19.233477 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:58:19.233477 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:58:19.233477 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:58:19.233477 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:58:19.233477 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:58:19.233477 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 05:58:19.233477 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 05:58:19.233477 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 05:58:19.234400 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 05:58:19.234400 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:58:19.234400 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:58:19.234400 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:58:19.234400 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:58:19.234400 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:58:19.234400 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:58:19.234400 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:58:19.234400 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 24 05:58:19.234400 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:58:19.234400 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:58:19.234400 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:58:19.234400 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:58:19.234400 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:58:19.234400 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:58:19.234400 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:58:19.234400 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:58:19.235286 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 05:58:19.235286 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 05:58:19.235286 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 05:58:19.235286 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 05:58:19.235286 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 05:58:19.235286 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 05:58:19.235286 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 05:58:19.235286 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 24 05:58:19.235286 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 448/903] 10.222.0.62 () {56 vars in 1111 bytes} [Wed Aug 24 05:58:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:58:19.240623 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477c71a9-49a7-4752-be88-e04f24b5657b None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:19.309109 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 456/904] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Aug 24 05:58:19 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 05:58:19.343371 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:19.359025 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477c71a9-49a7-4752-be88-e04f24b5657b None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:19.393702 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612cc4f0-397c-4b1b-be9f-1bd9b227a6fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:19.394317 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612cc4f0-397c-4b1b-be9f-1bd9b227a6fa None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:19.394596 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612cc4f0-397c-4b1b-be9f-1bd9b227a6fa None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:19.394871 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612cc4f0-397c-4b1b-be9f-1bd9b227a6fa None admin] PATH_INFO: `/v3/projects` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:19.395558 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612cc4f0-397c-4b1b-be9f-1bd9b227a6fa None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:58:19.395863 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612cc4f0-397c-4b1b-be9f-1bd9b227a6fa None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:19.397337 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612cc4f0-397c-4b1b-be9f-1bd9b227a6fa None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:19.409931 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 449/905] 10.222.0.62 () {56 vars in 1148 bytes} [Wed Aug 24 05:58:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:58:19.415701 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5437a562-7303-4a76-a6bc-ac939e51331e None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:19.462289 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612cc4f0-397c-4b1b-be9f-1bd9b227a6fa None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:19.507701 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:19.531685 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5437a562-7303-4a76-a6bc-ac939e51331e None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:19.570284 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be9c92e-30fe-4576-837a-1815c5233a5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:19.570897 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be9c92e-30fe-4576-837a-1815c5233a5f None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:19.571173 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be9c92e-30fe-4576-837a-1815c5233a5f None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:19.571515 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be9c92e-30fe-4576-837a-1815c5233a5f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:19.572146 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be9c92e-30fe-4576-837a-1815c5233a5f None admin] RBAC: Adding query filter params (role.id=2499f7e98f3e498ab329637f00bb2da4, scope.project.id=181f4762037e454489d9552cc603179c, user.id=b82726a12436435cb931b986ef884c2e) {{(pid=81851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:58:19.572447 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be9c92e-30fe-4576-837a-1815c5233a5f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:19.573895 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be9c92e-30fe-4576-837a-1815c5233a5f None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:19.574427 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612cc4f0-397c-4b1b-be9f-1bd9b227a6fa None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:19.588665 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 457/906] 10.222.0.62 () {56 vars in 1374 bytes} [Wed Aug 24 05:58:19 2022] GET /identity/v3/role_assignments?user.id=b82726a12436435cb931b986ef884c2e&scope.project.id=181f4762037e454489d9552cc603179c&role.id=2499f7e98f3e498ab329637f00bb2da4 => generated 259 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:58:19.591604 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be9c92e-30fe-4576-837a-1815c5233a5f None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:19.614319 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c53277-8bd9-4dbd-841c-f8dbd2a65e26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:19.615365 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c53277-8bd9-4dbd-841c-f8dbd2a65e26 None admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:19.615859 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c53277-8bd9-4dbd-841c-f8dbd2a65e26 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:19.616327 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c53277-8bd9-4dbd-841c-f8dbd2a65e26 None admin] PATH_INFO: `/v3/services` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:19.617212 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c53277-8bd9-4dbd-841c-f8dbd2a65e26 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:19.618963 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c53277-8bd9-4dbd-841c-f8dbd2a65e26 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:19.707048 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be9c92e-30fe-4576-837a-1815c5233a5f None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:19.717326 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 450/907] 10.222.0.62 () {60 vars in 1141 bytes} [Wed Aug 24 05:58:19 2022] POST /identity/v3/services => generated 241 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:58:19.756641 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2927cf-6e72-4597-a1c1-893a78e26c52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:19.757287 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2927cf-6e72-4597-a1c1-893a78e26c52 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:19.757565 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2927cf-6e72-4597-a1c1-893a78e26c52 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:19.757887 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2927cf-6e72-4597-a1c1-893a78e26c52 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:19.758423 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2927cf-6e72-4597-a1c1-893a78e26c52 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:19.759905 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2927cf-6e72-4597-a1c1-893a78e26c52 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:19.764039 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:19.771075 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application [None req-6c2927cf-6e72-4597-a1c1-893a78e26c52 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 05:58:19.771075 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:58:19.771075 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 05:58:19.771075 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 05:58:19.771075 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 05:58:19.771075 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 05:58:19.771075 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 05:58:19.771075 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 05:58:19.771075 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 05:58:19.771075 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Aug 24 05:58:19.771075 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 05:58:19.771075 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Aug 24 05:58:19.771075 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:58:19.771075 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:58:19.771075 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:58:19.771075 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:58:19.772047 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:58:19.772047 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:58:19.772047 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:58:19.772047 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:58:19.772047 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:58:19.772047 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:58:19.772047 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:58:19.772047 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 05:58:19.772047 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 05:58:19.772047 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 05:58:19.772047 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 05:58:19.772047 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:58:19.772047 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:58:19.772047 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:58:19.772047 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:58:19.772047 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:58:19.772047 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:58:19.773011 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:58:19.773011 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application with Lock( Aug 24 05:58:19.773011 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:58:19.773011 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:58:19.773011 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:58:19.773011 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:58:19.773011 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:58:19.773011 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:58:19.773011 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:58:19.773011 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:58:19.773011 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 05:58:19.773011 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 05:58:19.773011 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 05:58:19.773011 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Aug 24 05:58:19.773011 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 458/908] 10.222.0.62 () {56 vars in 1132 bytes} [Wed Aug 24 05:58:19 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 05:58:19.779901 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c53277-8bd9-4dbd-841c-f8dbd2a65e26 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:19.784404 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:19.894660 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c53277-8bd9-4dbd-841c-f8dbd2a65e26 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:19.930652 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57ca3c9-6e74-4924-8574-2d98076c605b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:19.931267 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57ca3c9-6e74-4924-8574-2d98076c605b None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:19.931543 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57ca3c9-6e74-4924-8574-2d98076c605b None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:19.931878 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57ca3c9-6e74-4924-8574-2d98076c605b None admin] PATH_INFO: `/v3/services` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:19.932529 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57ca3c9-6e74-4924-8574-2d98076c605b None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:58:19.932827 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57ca3c9-6e74-4924-8574-2d98076c605b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:19.934439 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57ca3c9-6e74-4924-8574-2d98076c605b None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:19.942019 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 451/909] 10.222.0.62 () {56 vars in 1126 bytes} [Wed Aug 24 05:58:19 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:58:19.948371 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2927cf-6e72-4597-a1c1-893a78e26c52 None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:20.054177 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2927cf-6e72-4597-a1c1-893a78e26c52 None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:20.090385 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e975ebc2-2e79-471b-ab4f-26f4f3583b9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:20.090990 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e975ebc2-2e79-471b-ab4f-26f4f3583b9d None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:20.091264 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e975ebc2-2e79-471b-ab4f-26f4f3583b9d None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:20.091532 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e975ebc2-2e79-471b-ab4f-26f4f3583b9d None admin] PATH_INFO: `/v3/services` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:20.092783 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e975ebc2-2e79-471b-ab4f-26f4f3583b9d None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:58:20.093136 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e975ebc2-2e79-471b-ab4f-26f4f3583b9d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:20.094613 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e975ebc2-2e79-471b-ab4f-26f4f3583b9d None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:20.099970 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 459/910] 10.222.0.62 () {56 vars in 1126 bytes} [Wed Aug 24 05:58:19 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:58:20.106130 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57ca3c9-6e74-4924-8574-2d98076c605b None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:20.176993 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:20.214840 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57ca3c9-6e74-4924-8574-2d98076c605b None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:20.250823 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ff32ac-e45a-4819-a5e8-03d279d83a90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:20.251797 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ff32ac-e45a-4819-a5e8-03d279d83a90 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:20.252325 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ff32ac-e45a-4819-a5e8-03d279d83a90 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:20.252814 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ff32ac-e45a-4819-a5e8-03d279d83a90 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:20.253708 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ff32ac-e45a-4819-a5e8-03d279d83a90 None admin] RBAC: Adding query filter params (interface=public, service_id=51cc626b1b7144a3897c45849183128a, region_id=RegionOne) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:58:20.254214 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ff32ac-e45a-4819-a5e8-03d279d83a90 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:20.255777 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ff32ac-e45a-4819-a5e8-03d279d83a90 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:20.268590 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 452/911] 10.222.0.62 () {56 vars in 1251 bytes} [Wed Aug 24 05:58:20 2022] GET /identity/v3/endpoints?service_id=51cc626b1b7144a3897c45849183128a&interface=public®ion_id=RegionOne => generated 194 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:58:20.270366 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e975ebc2-2e79-471b-ab4f-26f4f3583b9d None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:20.343267 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:20.383308 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e975ebc2-2e79-471b-ab4f-26f4f3583b9d None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:20.418289 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424a5edc-e2fb-4af3-9fa2-bc33f0ef8dfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:20.418892 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424a5edc-e2fb-4af3-9fa2-bc33f0ef8dfc None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:20.419167 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424a5edc-e2fb-4af3-9fa2-bc33f0ef8dfc None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:20.419438 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424a5edc-e2fb-4af3-9fa2-bc33f0ef8dfc None admin] PATH_INFO: `/v3/services` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:20.420476 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424a5edc-e2fb-4af3-9fa2-bc33f0ef8dfc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:20.421858 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424a5edc-e2fb-4af3-9fa2-bc33f0ef8dfc None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:20.428370 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 460/912] 10.222.0.62 () {56 vars in 1087 bytes} [Wed Aug 24 05:58:20 2022] GET /identity/v3/services => generated 1497 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:58:21.036621 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0ac1a1-0e6e-482e-a882-3afde20f554b None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:21.037613 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0ac1a1-0e6e-482e-a882-3afde20f554b None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:21.038130 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0ac1a1-0e6e-482e-a882-3afde20f554b None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:21.039922 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 453/913] 10.222.0.62 () {54 vars in 927 bytes} [Wed Aug 24 05:58:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:58:21.045931 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057103e9-8b2d-4cfc-9720-f4195b963861 None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:21.046440 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057103e9-8b2d-4cfc-9720-f4195b963861 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:21.046915 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057103e9-8b2d-4cfc-9720-f4195b963861 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:21.065670 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-057103e9-8b2d-4cfc-9720-f4195b963861 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:58:21.067157 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-057103e9-8b2d-4cfc-9720-f4195b963861 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:58:21.141453 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 461/914] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Aug 24 05:58:21 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 05:58:21.408709 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0ac1a1-0e6e-482e-a882-3afde20f554b None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:21.516157 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92aca0d-a447-4f89-8f3f-1688989b43e6 None None] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:21.516752 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92aca0d-a447-4f89-8f3f-1688989b43e6 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:21.516752 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92aca0d-a447-4f89-8f3f-1688989b43e6 None None] PATH_INFO: `/` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:21.518045 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 462/915] 10.222.0.62 () {54 vars in 927 bytes} [Wed Aug 24 05:58:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 05:58:21.522805 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6407c921-2386-4e44-99e8-10a72e847a8e None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:21.523081 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6407c921-2386-4e44-99e8-10a72e847a8e None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:21.523354 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6407c921-2386-4e44-99e8-10a72e847a8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:21.524087 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0ac1a1-0e6e-482e-a882-3afde20f554b None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:21.542966 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-6407c921-2386-4e44-99e8-10a72e847a8e None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:58:21.544127 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-6407c921-2386-4e44-99e8-10a72e847a8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:58:21.560285 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:21.570820 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2be389-7be6-4f21-a3a2-6b1c012e3977 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:21.571606 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2be389-7be6-4f21-a3a2-6b1c012e3977 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:21.574392 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2be389-7be6-4f21-a3a2-6b1c012e3977 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:21.574392 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2be389-7be6-4f21-a3a2-6b1c012e3977 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:21.574392 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2be389-7be6-4f21-a3a2-6b1c012e3977 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:21.574392 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2be389-7be6-4f21-a3a2-6b1c012e3977 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:21.584034 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-ae2be389-7be6-4f21-a3a2-6b1c012e3977 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 05:58:21.584034 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:58:21.584034 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 05:58:21.584034 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 05:58:21.584034 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 05:58:21.584034 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 05:58:21.584034 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 05:58:21.584034 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 05:58:21.584034 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 05:58:21.584034 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 24 05:58:21.584034 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 05:58:21.584034 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 24 05:58:21.584034 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:58:21.584034 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:58:21.584034 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:58:21.584034 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:58:21.584034 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:58:21.585294 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:58:21.585294 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:58:21.585294 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:58:21.585294 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:58:21.585294 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:58:21.585294 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:58:21.585294 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 05:58:21.585294 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 05:58:21.585294 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 05:58:21.585294 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 05:58:21.585294 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:58:21.585294 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:58:21.585294 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:58:21.585294 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:58:21.585294 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:58:21.585294 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:58:21.586305 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:58:21.586305 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 24 05:58:21.586305 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:58:21.586305 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:58:21.586305 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:58:21.586305 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:58:21.586305 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:58:21.586305 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:58:21.586305 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:58:21.586305 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:58:21.586305 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 05:58:21.586305 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 05:58:21.586305 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 05:58:21.586305 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 24 05:58:21.587810 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 454/916] 10.222.0.62 () {56 vars in 1114 bytes} [Wed Aug 24 05:58:21 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:58:21.594209 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2be389-7be6-4f21-a3a2-6b1c012e3977 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:21.620133 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:21.644813 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 463/917] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Aug 24 05:58:21 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 05:58:21.705958 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2be389-7be6-4f21-a3a2-6b1c012e3977 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:21.740682 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8f713a-961e-41ce-957d-6d8a4f535046 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:21.741383 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8f713a-961e-41ce-957d-6d8a4f535046 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:21.741732 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8f713a-961e-41ce-957d-6d8a4f535046 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:21.742074 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8f713a-961e-41ce-957d-6d8a4f535046 None admin] PATH_INFO: `/v3/services` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:21.742765 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8f713a-961e-41ce-957d-6d8a4f535046 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:58:21.743136 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8f713a-961e-41ce-957d-6d8a4f535046 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:21.744713 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8f713a-961e-41ce-957d-6d8a4f535046 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:21.750813 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 455/918] 10.222.0.62 () {56 vars in 1114 bytes} [Wed Aug 24 05:58:21 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:58:21.756430 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6407c921-2386-4e44-99e8-10a72e847a8e None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:21.842811 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dd5a02-7d01-4b3d-ace2-8da36041fc5c None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:21.843087 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dd5a02-7d01-4b3d-ace2-8da36041fc5c None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:21.843363 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dd5a02-7d01-4b3d-ace2-8da36041fc5c None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:21.845072 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 456/919] 10.222.0.62 () {54 vars in 927 bytes} [Wed Aug 24 05:58:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 05:58:21.850963 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447301ea-eda2-4897-9bbb-e360b4135f60 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:21.851239 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447301ea-eda2-4897-9bbb-e360b4135f60 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:21.851512 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447301ea-eda2-4897-9bbb-e360b4135f60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:21.866603 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6407c921-2386-4e44-99e8-10a72e847a8e None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:21.869311 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-447301ea-eda2-4897-9bbb-e360b4135f60 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:58:21.870478 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-447301ea-eda2-4897-9bbb-e360b4135f60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:58:21.912415 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bb6d24-60d7-41cb-8b7d-13a899c1ec16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:21.913413 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bb6d24-60d7-41cb-8b7d-13a899c1ec16 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:21.914444 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bb6d24-60d7-41cb-8b7d-13a899c1ec16 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:21.914925 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bb6d24-60d7-41cb-8b7d-13a899c1ec16 None admin] PATH_INFO: `/v3/services` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:21.915781 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8bb6d24-60d7-41cb-8b7d-13a899c1ec16 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:58:21.916811 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8bb6d24-60d7-41cb-8b7d-13a899c1ec16 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:21.919121 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8bb6d24-60d7-41cb-8b7d-13a899c1ec16 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:21.927065 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 464/920] 10.222.0.62 () {56 vars in 1114 bytes} [Wed Aug 24 05:58:21 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:58:21.933591 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bb6d24-60d7-41cb-8b7d-13a899c1ec16 None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:21.934930 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:21.948052 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 457/921] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Aug 24 05:58:21 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 05:58:21.949113 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447301ea-eda2-4897-9bbb-e360b4135f60 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:22.005063 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:22.057520 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447301ea-eda2-4897-9bbb-e360b4135f60 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:22.077661 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bb6d24-60d7-41cb-8b7d-13a899c1ec16 None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:22.094799 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7575436e-9a03-4c5f-a607-cbe01fc585fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:22.095562 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7575436e-9a03-4c5f-a607-cbe01fc585fc None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:22.095930 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7575436e-9a03-4c5f-a607-cbe01fc585fc None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:22.096451 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7575436e-9a03-4c5f-a607-cbe01fc585fc None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:22.107012 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7575436e-9a03-4c5f-a607-cbe01fc585fc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:58:22.112401 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-7575436e-9a03-4c5f-a607-cbe01fc585fc None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 05:58:22.112401 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:58:22.112401 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:58:22.112401 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:58:22.112401 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:58:22.112401 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:58:22.112401 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:58:22.112401 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:58:22.112401 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:58:22.112401 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:58:22.112401 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:58:22.112401 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:58:22.112401 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:58:22.112401 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:58:22.112401 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:58:22.112401 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:58:22.113523 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:58:22.113523 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:58:22.113523 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:58:22.113523 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:58:22.113523 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:58:22.113523 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:58:22.113523 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:58:22.113523 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:58:22.113523 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:58:22.113523 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:58:22.113523 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:58:22.113523 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 24 05:58:22.113523 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:58:22.113523 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:58:22.113523 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:58:22.113523 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:58:22.114550 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:58:22.114550 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:58:22.114550 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:58:22.114550 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:58:22.114550 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:58:22.114550 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:58:22.114550 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:58:22.114550 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:58:22.114550 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:58:22.114550 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:58:22.114550 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 05:58:22.114550 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 24 05:58:22.116469 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 458/922] 10.222.0.62 () {56 vars in 1120 bytes} [Wed Aug 24 05:58:21 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:58:22.121728 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7575436e-9a03-4c5f-a607-cbe01fc585fc None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:22.131264 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c71d23-15c8-4508-9976-4d11068c173c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:22.132191 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c71d23-15c8-4508-9976-4d11068c173c None admin] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:22.132703 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c71d23-15c8-4508-9976-4d11068c173c None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:22.133169 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c71d23-15c8-4508-9976-4d11068c173c None admin] PATH_INFO: `/v3/endpoints` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:22.134026 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c71d23-15c8-4508-9976-4d11068c173c None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:22.136042 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c71d23-15c8-4508-9976-4d11068c173c None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:22.179502 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:22.207299 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 465/923] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Aug 24 05:58:21 2022] POST /identity/v3/endpoints => generated 346 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:58:22.228841 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c71d23-15c8-4508-9976-4d11068c173c None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:22.239492 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7575436e-9a03-4c5f-a607-cbe01fc585fc None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:22.286037 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70757e58-6765-4a0c-9926-3f47d7beddd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:22.286827 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70757e58-6765-4a0c-9926-3f47d7beddd9 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:22.286827 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70757e58-6765-4a0c-9926-3f47d7beddd9 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:22.287094 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70757e58-6765-4a0c-9926-3f47d7beddd9 None admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:22.287778 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70757e58-6765-4a0c-9926-3f47d7beddd9 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:58:22.288078 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70757e58-6765-4a0c-9926-3f47d7beddd9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:22.289521 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70757e58-6765-4a0c-9926-3f47d7beddd9 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:22.299584 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 459/924] 10.222.0.62 () {56 vars in 1115 bytes} [Wed Aug 24 05:58:22 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:58:22.305172 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70757e58-6765-4a0c-9926-3f47d7beddd9 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:22.352634 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c71d23-15c8-4508-9976-4d11068c173c None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:22.363016 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:22.387769 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c2f315-b062-45c2-a2be-1ad26380e4b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:22.388960 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c2f315-b062-45c2-a2be-1ad26380e4b5 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:22.389215 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c2f315-b062-45c2-a2be-1ad26380e4b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:22.389489 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c2f315-b062-45c2-a2be-1ad26380e4b5 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:22.390075 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c2f315-b062-45c2-a2be-1ad26380e4b5 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:22.391455 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c2f315-b062-45c2-a2be-1ad26380e4b5 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:22.400650 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application [None req-69c2f315-b062-45c2-a2be-1ad26380e4b5 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 05:58:22.400650 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:58:22.400650 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 05:58:22.400650 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 05:58:22.400650 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 05:58:22.400650 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 05:58:22.400650 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 05:58:22.400650 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 05:58:22.400650 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 05:58:22.400650 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Aug 24 05:58:22.400650 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 05:58:22.400650 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Aug 24 05:58:22.400650 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:58:22.400650 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:58:22.400650 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:58:22.400650 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:58:22.400650 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:58:22.401828 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:58:22.401828 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:58:22.401828 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:58:22.401828 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:58:22.401828 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:58:22.401828 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:58:22.401828 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 05:58:22.401828 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 05:58:22.401828 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 05:58:22.401828 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 05:58:22.401828 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:58:22.401828 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:58:22.401828 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:58:22.401828 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:58:22.401828 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:58:22.401828 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:58:22.402664 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:58:22.402664 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application with Lock( Aug 24 05:58:22.402664 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:58:22.402664 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:58:22.402664 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:58:22.402664 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:58:22.402664 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:58:22.402664 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:58:22.402664 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:58:22.402664 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:58:22.402664 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 05:58:22.402664 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 05:58:22.402664 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 05:58:22.402664 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Aug 24 05:58:22.402664 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 466/925] 10.222.0.62 () {56 vars in 1126 bytes} [Wed Aug 24 05:58:22 2022] GET /identity/v3/services/object-store => generated 93 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:58:22.404857 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f620c827-ce02-49ab-8388-706d87cca042 None None] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:22.404857 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f620c827-ce02-49ab-8388-706d87cca042 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:22.405850 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f620c827-ce02-49ab-8388-706d87cca042 None None] PATH_INFO: `/` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:22.407589 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 467/926] 10.222.0.62 () {54 vars in 927 bytes} [Wed Aug 24 05:58:22 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:58:22.408372 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f620c827-ce02-49ab-8388-706d87cca042 None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:22.429676 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70757e58-6765-4a0c-9926-3f47d7beddd9 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:22.440346 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:22.465483 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a1ff61-de20-4f4a-a5be-4d6cc82a0af1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:22.466168 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a1ff61-de20-4f4a-a5be-4d6cc82a0af1 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:22.466448 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a1ff61-de20-4f4a-a5be-4d6cc82a0af1 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:22.466745 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a1ff61-de20-4f4a-a5be-4d6cc82a0af1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:22.478160 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a1ff61-de20-4f4a-a5be-4d6cc82a0af1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:22.479609 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a1ff61-de20-4f4a-a5be-4d6cc82a0af1 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:22.496722 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-e3a1ff61-de20-4f4a-a5be-4d6cc82a0af1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:58:22.496722 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:58:22.496722 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:58:22.496722 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:58:22.496722 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:58:22.496722 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:58:22.496722 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:58:22.496722 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:58:22.496722 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:58:22.496722 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:58:22.496722 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:58:22.496722 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:58:22.496722 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 05:58:22.496722 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 05:58:22.496722 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 05:58:22.496722 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 05:58:22.497691 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:58:22.497691 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:58:22.497691 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:58:22.497691 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:58:22.497691 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:58:22.497691 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:58:22.497691 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:58:22.497691 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 24 05:58:22.497691 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:58:22.497691 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:58:22.497691 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:58:22.497691 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:58:22.497691 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:58:22.497691 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:58:22.497691 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:58:22.497691 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:58:22.497691 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 05:58:22.499101 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 05:58:22.499101 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:58:22.499101 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 24 05:58:22.500541 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 460/927] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Aug 24 05:58:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:58:22.501856 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5608adc5-9d21-4a4a-bfe6-e925ddf9db8d None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:22.502270 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5608adc5-9d21-4a4a-bfe6-e925ddf9db8d None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:22.502669 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5608adc5-9d21-4a4a-bfe6-e925ddf9db8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:22.521060 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-5608adc5-9d21-4a4a-bfe6-e925ddf9db8d None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:58:22.522240 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-5608adc5-9d21-4a4a-bfe6-e925ddf9db8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:58:22.523149 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f620c827-ce02-49ab-8388-706d87cca042 None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:22.560206 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f4d6fb-31b9-4a4a-bb70-6ff15149573c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:22.561280 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f4d6fb-31b9-4a4a-bb70-6ff15149573c None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:22.561562 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f4d6fb-31b9-4a4a-bb70-6ff15149573c None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:22.561835 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f4d6fb-31b9-4a4a-bb70-6ff15149573c None admin] PATH_INFO: `/v3/services` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:22.562459 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f4d6fb-31b9-4a4a-bb70-6ff15149573c None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:58:22.562763 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f4d6fb-31b9-4a4a-bb70-6ff15149573c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:22.564255 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f4d6fb-31b9-4a4a-bb70-6ff15149573c None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:22.569148 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 468/928] 10.222.0.62 () {56 vars in 1122 bytes} [Wed Aug 24 05:58:22 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:58:22.569936 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f4d6fb-31b9-4a4a-bb70-6ff15149573c None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:22.594500 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 461/929] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Aug 24 05:58:22 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 05:58:22.595627 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5608adc5-9d21-4a4a-bfe6-e925ddf9db8d None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:22.628769 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:22.695843 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:22.714577 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5608adc5-9d21-4a4a-bfe6-e925ddf9db8d None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:22.727168 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f4d6fb-31b9-4a4a-bb70-6ff15149573c None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:22.754185 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b339967-da47-4a25-a578-4ec3e0853da1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:22.755181 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b339967-da47-4a25-a578-4ec3e0853da1 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:22.755663 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b339967-da47-4a25-a578-4ec3e0853da1 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:22.756160 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b339967-da47-4a25-a578-4ec3e0853da1 None admin] PATH_INFO: `/v3/services` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:22.757144 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b339967-da47-4a25-a578-4ec3e0853da1 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:58:22.757741 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b339967-da47-4a25-a578-4ec3e0853da1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:22.759367 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b339967-da47-4a25-a578-4ec3e0853da1 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:22.765750 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 462/930] 10.222.0.62 () {56 vars in 1122 bytes} [Wed Aug 24 05:58:22 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:58:22.771513 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b339967-da47-4a25-a578-4ec3e0853da1 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:22.774227 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e268b886-e84f-4f71-be8b-35b8e6546cf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:22.774863 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e268b886-e84f-4f71-be8b-35b8e6546cf0 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:22.775143 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e268b886-e84f-4f71-be8b-35b8e6546cf0 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:22.775408 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e268b886-e84f-4f71-be8b-35b8e6546cf0 None admin] PATH_INFO: `/v3/domains` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:22.776044 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e268b886-e84f-4f71-be8b-35b8e6546cf0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:58:22.776340 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e268b886-e84f-4f71-be8b-35b8e6546cf0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:22.777730 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e268b886-e84f-4f71-be8b-35b8e6546cf0 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:22.799223 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 469/931] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Aug 24 05:58:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:58:22.803397 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e268b886-e84f-4f71-be8b-35b8e6546cf0 None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:22.903048 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:22.906486 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b339967-da47-4a25-a578-4ec3e0853da1 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:22.909507 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e268b886-e84f-4f71-be8b-35b8e6546cf0 None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:22.944760 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90081bd0-fb1a-45ab-b920-7d01d9f118ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:22.945344 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90081bd0-fb1a-45ab-b920-7d01d9f118ed None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:22.945626 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90081bd0-fb1a-45ab-b920-7d01d9f118ed None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:22.945899 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90081bd0-fb1a-45ab-b920-7d01d9f118ed None admin] PATH_INFO: `/v3/endpoints` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:22.946477 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8311c81-1c92-43f6-bd8d-55a931810ac4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:22.946593 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90081bd0-fb1a-45ab-b920-7d01d9f118ed None admin] RBAC: Adding query filter params (interface=public, service_id=c2fd3d0526d24c39a9cd5fb18573dd75, region_id=RegionOne) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:58:22.946885 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90081bd0-fb1a-45ab-b920-7d01d9f118ed None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:22.947149 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8311c81-1c92-43f6-bd8d-55a931810ac4 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:22.947425 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8311c81-1c92-43f6-bd8d-55a931810ac4 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:22.947691 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8311c81-1c92-43f6-bd8d-55a931810ac4 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:22.948308 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90081bd0-fb1a-45ab-b920-7d01d9f118ed None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:22.953848 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 463/932] 10.222.0.62 () {56 vars in 1251 bytes} [Wed Aug 24 05:58:22 2022] GET /identity/v3/endpoints?service_id=c2fd3d0526d24c39a9cd5fb18573dd75&interface=public®ion_id=RegionOne => generated 194 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:58:22.954466 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90081bd0-fb1a-45ab-b920-7d01d9f118ed None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:22.959857 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8311c81-1c92-43f6-bd8d-55a931810ac4 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:22.961373 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8311c81-1c92-43f6-bd8d-55a931810ac4 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:22.972821 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application [None req-c8311c81-1c92-43f6-bd8d-55a931810ac4 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 05:58:22.972821 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:58:22.972821 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:58:22.972821 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:58:22.972821 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:58:22.972821 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:58:22.972821 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:58:22.972821 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:58:22.972821 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:58:22.972821 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:58:22.972821 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:58:22.972821 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:58:22.972821 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 05:58:22.972821 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 05:58:22.972821 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 05:58:22.972821 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 05:58:22.973769 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:58:22.973769 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:58:22.973769 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 05:58:22.973769 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 05:58:22.973769 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 05:58:22.973769 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 05:58:22.973769 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:58:22.973769 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:58:22.973769 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:58:22.973769 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:58:22.973769 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:58:22.973769 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application with Lock( Aug 24 05:58:22.973769 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:58:22.973769 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:58:22.973769 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:58:22.973769 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:58:22.973769 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:58:22.973769 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:58:22.974711 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:58:22.974711 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:58:22.974711 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 05:58:22.974711 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 05:58:22.974711 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 05:58:22.974711 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 05:58:22.974711 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 05:58:22.974711 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 05:58:22.974711 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 05:58:22.974711 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Aug 24 05:58:22.974711 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 470/933] 10.222.0.62 () {56 vars in 1117 bytes} [Wed Aug 24 05:58:22 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:58:22.974711 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8311c81-1c92-43f6-bd8d-55a931810ac4 None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:22.976189 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:23.062123 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90081bd0-fb1a-45ab-b920-7d01d9f118ed None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:23.080770 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8311c81-1c92-43f6-bd8d-55a931810ac4 None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:23.098237 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6192cb4-7abb-44f9-bff6-e575d921ab5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:23.098859 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6192cb4-7abb-44f9-bff6-e575d921ab5f None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:23.099133 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6192cb4-7abb-44f9-bff6-e575d921ab5f None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:23.099399 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6192cb4-7abb-44f9-bff6-e575d921ab5f None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:23.099971 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6192cb4-7abb-44f9-bff6-e575d921ab5f None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:23.101363 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6192cb4-7abb-44f9-bff6-e575d921ab5f None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:23.110744 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-c6192cb4-7abb-44f9-bff6-e575d921ab5f None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 05:58:23.110744 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:58:23.110744 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 05:58:23.110744 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 05:58:23.110744 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 05:58:23.110744 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 05:58:23.110744 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 05:58:23.110744 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 05:58:23.110744 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 05:58:23.110744 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 24 05:58:23.110744 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 05:58:23.110744 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 24 05:58:23.110744 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:58:23.110744 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:58:23.110744 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:58:23.110744 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:58:23.111748 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:58:23.111748 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:58:23.111748 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:58:23.111748 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:58:23.111748 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:58:23.111748 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:58:23.111748 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:58:23.111748 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 05:58:23.111748 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 05:58:23.111748 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 05:58:23.111748 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 05:58:23.111748 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:58:23.111748 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:58:23.111748 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:58:23.111748 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:58:23.111748 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:58:23.111748 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:58:23.112687 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:58:23.112687 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 24 05:58:23.112687 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:58:23.112687 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:58:23.112687 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:58:23.112687 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:58:23.112687 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:58:23.112687 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:58:23.112687 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:58:23.112687 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:58:23.112687 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 05:58:23.112687 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 05:58:23.112687 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 05:58:23.112687 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 24 05:58:23.112687 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 464/934] 10.222.0.62 () {56 vars in 1132 bytes} [Wed Aug 24 05:58:22 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:58:23.113518 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6192cb4-7abb-44f9-bff6-e575d921ab5f None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:23.120990 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3221fb09-c9f6-4197-883e-b1630dbd4166 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:23.121609 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3221fb09-c9f6-4197-883e-b1630dbd4166 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:23.121892 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3221fb09-c9f6-4197-883e-b1630dbd4166 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:23.122169 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3221fb09-c9f6-4197-883e-b1630dbd4166 None admin] PATH_INFO: `/v3/services` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:23.122846 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3221fb09-c9f6-4197-883e-b1630dbd4166 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:23.124774 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3221fb09-c9f6-4197-883e-b1630dbd4166 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:23.132226 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 471/935] 10.222.0.62 () {56 vars in 1087 bytes} [Wed Aug 24 05:58:22 2022] GET /identity/v3/services => generated 1497 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:58:23.133076 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3221fb09-c9f6-4197-883e-b1630dbd4166 None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:23.159598 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:23.228960 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:23.260187 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6192cb4-7abb-44f9-bff6-e575d921ab5f None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:23.261851 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3221fb09-c9f6-4197-883e-b1630dbd4166 None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:23.311724 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34be328d-82b8-48c5-9b85-205c033a0cc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:23.312044 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30649e8-5626-4f08-8668-c34cfd4fd036 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:23.312345 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34be328d-82b8-48c5-9b85-205c033a0cc1 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:23.312456 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30649e8-5626-4f08-8668-c34cfd4fd036 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:23.312694 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34be328d-82b8-48c5-9b85-205c033a0cc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:23.312808 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30649e8-5626-4f08-8668-c34cfd4fd036 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:23.312961 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30649e8-5626-4f08-8668-c34cfd4fd036 None admin] PATH_INFO: `/v3/users` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:23.313058 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34be328d-82b8-48c5-9b85-205c033a0cc1 None admin] PATH_INFO: `/v3/services` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:23.313599 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34be328d-82b8-48c5-9b85-205c033a0cc1 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:58:23.313698 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30649e8-5626-4f08-8668-c34cfd4fd036 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:58:23.313906 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34be328d-82b8-48c5-9b85-205c033a0cc1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:23.314007 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30649e8-5626-4f08-8668-c34cfd4fd036 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:23.315408 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34be328d-82b8-48c5-9b85-205c033a0cc1 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:23.315509 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30649e8-5626-4f08-8668-c34cfd4fd036 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:23.323248 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 472/936] 10.222.0.62 () {56 vars in 1126 bytes} [Wed Aug 24 05:58:23 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:58:23.330161 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34be328d-82b8-48c5-9b85-205c033a0cc1 None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:23.343073 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 465/937] 10.222.0.62 () {56 vars in 1149 bytes} [Wed Aug 24 05:58:23 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:58:23.349509 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30649e8-5626-4f08-8668-c34cfd4fd036 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:23.424007 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:23.440037 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34be328d-82b8-48c5-9b85-205c033a0cc1 None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:23.477658 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30649e8-5626-4f08-8668-c34cfd4fd036 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:23.481148 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3680b3d-484d-41c0-aadd-a3ac54d4d833 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:23.481760 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3680b3d-484d-41c0-aadd-a3ac54d4d833 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:23.482037 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3680b3d-484d-41c0-aadd-a3ac54d4d833 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:23.482856 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3680b3d-484d-41c0-aadd-a3ac54d4d833 None admin] PATH_INFO: `/v3/services` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:23.483485 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3680b3d-484d-41c0-aadd-a3ac54d4d833 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:58:23.483788 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3680b3d-484d-41c0-aadd-a3ac54d4d833 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:23.485207 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3680b3d-484d-41c0-aadd-a3ac54d4d833 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:23.495162 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 473/938] 10.222.0.62 () {56 vars in 1126 bytes} [Wed Aug 24 05:58:23 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:58:23.496174 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3680b3d-484d-41c0-aadd-a3ac54d4d833 None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:23.526197 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47b6d77-f6c7-43da-a78a-30cc9da04fa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:23.527966 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47b6d77-f6c7-43da-a78a-30cc9da04fa7 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:23.528610 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47b6d77-f6c7-43da-a78a-30cc9da04fa7 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:23.529748 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47b6d77-f6c7-43da-a78a-30cc9da04fa7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:23.540824 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:23.544748 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47b6d77-f6c7-43da-a78a-30cc9da04fa7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:23.547675 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47b6d77-f6c7-43da-a78a-30cc9da04fa7 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:23.567887 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-f47b6d77-f6c7-43da-a78a-30cc9da04fa7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:58:23.567887 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:58:23.567887 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:58:23.567887 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:58:23.567887 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:58:23.567887 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:58:23.567887 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:58:23.567887 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:58:23.567887 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:58:23.567887 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:58:23.567887 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:58:23.567887 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:58:23.567887 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 05:58:23.567887 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 05:58:23.567887 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 05:58:23.567887 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 05:58:23.569001 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:58:23.569001 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:58:23.569001 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:58:23.569001 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:58:23.569001 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:58:23.569001 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:58:23.569001 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:58:23.569001 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 24 05:58:23.569001 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:58:23.569001 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:58:23.569001 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:58:23.569001 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:58:23.569001 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:58:23.569001 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:58:23.569001 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:58:23.569001 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:58:23.569001 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 05:58:23.569987 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 05:58:23.569987 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:58:23.569987 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 24 05:58:23.572572 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 466/939] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Aug 24 05:58:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:58:23.575415 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47b6d77-f6c7-43da-a78a-30cc9da04fa7 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:23.611052 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3680b3d-484d-41c0-aadd-a3ac54d4d833 None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:23.647822 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd7c355-eaa9-44b6-8006-63e76da8f538 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:23.648536 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd7c355-eaa9-44b6-8006-63e76da8f538 None admin] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:23.649064 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd7c355-eaa9-44b6-8006-63e76da8f538 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:23.649442 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd7c355-eaa9-44b6-8006-63e76da8f538 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:23.650202 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd7c355-eaa9-44b6-8006-63e76da8f538 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:23.651813 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd7c355-eaa9-44b6-8006-63e76da8f538 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:23.713172 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47b6d77-f6c7-43da-a78a-30cc9da04fa7 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:23.749590 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45837e7e-14d7-4833-a341-6540520b89f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:23.750379 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45837e7e-14d7-4833-a341-6540520b89f1 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:23.750825 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45837e7e-14d7-4833-a341-6540520b89f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:23.751210 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45837e7e-14d7-4833-a341-6540520b89f1 None admin] PATH_INFO: `/v3/domains` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:23.751947 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45837e7e-14d7-4833-a341-6540520b89f1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:58:23.752347 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45837e7e-14d7-4833-a341-6540520b89f1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:23.754091 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45837e7e-14d7-4833-a341-6540520b89f1 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:23.769230 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 474/940] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Aug 24 05:58:23 2022] POST /identity/v3/endpoints => generated 347 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 05:58:23.775912 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 467/941] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Aug 24 05:58:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:58:23.782251 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd7c355-eaa9-44b6-8006-63e76da8f538 None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:23.786863 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:23.913047 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd7c355-eaa9-44b6-8006-63e76da8f538 None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:23.950209 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6157543-948c-473b-83ee-47db1d84092d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:23.951669 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6157543-948c-473b-83ee-47db1d84092d None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:23.952120 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6157543-948c-473b-83ee-47db1d84092d None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:23.952394 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6157543-948c-473b-83ee-47db1d84092d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:23.959501 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6157543-948c-473b-83ee-47db1d84092d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:23.960926 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6157543-948c-473b-83ee-47db1d84092d None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:23.972339 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application [None req-b6157543-948c-473b-83ee-47db1d84092d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 05:58:23.972339 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:58:23.972339 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:58:23.972339 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:58:23.972339 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:58:23.972339 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:58:23.972339 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:58:23.972339 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:58:23.972339 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:58:23.972339 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:58:23.972339 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:58:23.972339 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:58:23.972339 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 05:58:23.972339 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 05:58:23.972339 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 05:58:23.973352 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 05:58:23.973352 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:58:23.973352 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:58:23.973352 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:58:23.973352 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:58:23.973352 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:58:23.973352 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:58:23.973352 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:58:23.973352 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application with Lock( Aug 24 05:58:23.973352 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:58:23.973352 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:58:23.973352 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:58:23.973352 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:58:23.973352 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:58:23.973352 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:58:23.973352 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:58:23.973352 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:58:23.974253 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 05:58:23.974253 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 05:58:23.974253 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 05:58:23.974253 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 05:58:23.974253 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 05:58:23.974253 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 05:58:23.974253 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 05:58:23.974253 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Aug 24 05:58:23.974253 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 475/942] 10.222.0.62 () {56 vars in 1111 bytes} [Wed Aug 24 05:58:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 05:58:23.979632 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45837e7e-14d7-4833-a341-6540520b89f1 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:24.035150 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:24.083650 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45837e7e-14d7-4833-a341-6540520b89f1 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:24.118133 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9851af1-fa2a-4849-ae99-8fe0bd0ea359 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:24.118749 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9851af1-fa2a-4849-ae99-8fe0bd0ea359 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:24.119032 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9851af1-fa2a-4849-ae99-8fe0bd0ea359 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:24.119307 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9851af1-fa2a-4849-ae99-8fe0bd0ea359 None admin] PATH_INFO: `/v3/projects` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:24.119990 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9851af1-fa2a-4849-ae99-8fe0bd0ea359 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:58:24.120299 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9851af1-fa2a-4849-ae99-8fe0bd0ea359 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:24.121764 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9851af1-fa2a-4849-ae99-8fe0bd0ea359 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:24.134944 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 468/943] 10.222.0.62 () {56 vars in 1148 bytes} [Wed Aug 24 05:58:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:58:24.141454 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6157543-948c-473b-83ee-47db1d84092d None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:24.212072 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:24.214460 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c27ba9-1626-431d-8fac-4b045870a495 None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:24.215297 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c27ba9-1626-431d-8fac-4b045870a495 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:24.215783 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c27ba9-1626-431d-8fac-4b045870a495 None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:24.218906 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 469/944] 10.222.0.62 () {54 vars in 927 bytes} [Wed Aug 24 05:58:24 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:58:24.223231 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b163a25-32b0-45d4-b24d-cdbc078a4634 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:24.223778 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b163a25-32b0-45d4-b24d-cdbc078a4634 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:24.224260 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b163a25-32b0-45d4-b24d-cdbc078a4634 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:24.242494 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-6b163a25-32b0-45d4-b24d-cdbc078a4634 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:58:24.244094 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-6b163a25-32b0-45d4-b24d-cdbc078a4634 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:58:24.258660 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6157543-948c-473b-83ee-47db1d84092d None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:24.294259 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce933c76-67e1-47b8-8ae1-ea2d2ee674dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:24.295965 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce933c76-67e1-47b8-8ae1-ea2d2ee674dc None admin] REQUEST_METHOD: `PUT` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:24.296340 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce933c76-67e1-47b8-8ae1-ea2d2ee674dc None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:24.296846 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce933c76-67e1-47b8-8ae1-ea2d2ee674dc None admin] PATH_INFO: `/v3/projects/181f4762037e454489d9552cc603179c/users/b82726a12436435cb931b986ef884c2e/roles/2499f7e98f3e498ab329637f00bb2da4` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:24.299544 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce933c76-67e1-47b8-8ae1-ea2d2ee674dc None admin] RBAC: Authorizing `identity:create_grant(project_id=181f4762037e454489d9552cc603179c, user_id=b82726a12436435cb931b986ef884c2e, role_id=2499f7e98f3e498ab329637f00bb2da4)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:24.301940 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce933c76-67e1-47b8-8ae1-ea2d2ee674dc None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:24.323977 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 470/945] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Aug 24 05:58:24 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 05:58:24.394360 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 476/946] 10.222.0.62 () {58 vars in 1439 bytes} [Wed Aug 24 05:58:24 2022] PUT /identity/v3/projects/181f4762037e454489d9552cc603179c/users/b82726a12436435cb931b986ef884c2e/roles/2499f7e98f3e498ab329637f00bb2da4 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:58:24.591097 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b163a25-32b0-45d4-b24d-cdbc078a4634 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:24.635211 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:24.696618 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b163a25-32b0-45d4-b24d-cdbc078a4634 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:24.731962 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145053bf-9ce3-4e27-bbd4-680f15fd43fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:24.732632 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145053bf-9ce3-4e27-bbd4-680f15fd43fe None admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:24.732907 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145053bf-9ce3-4e27-bbd4-680f15fd43fe None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:24.733175 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145053bf-9ce3-4e27-bbd4-680f15fd43fe None admin] PATH_INFO: `/v3/projects` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:24.733806 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145053bf-9ce3-4e27-bbd4-680f15fd43fe None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:24.735225 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145053bf-9ce3-4e27-bbd4-680f15fd43fe None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:24.737828 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: WARNING keystone.server.flask.common [None req-145053bf-9ce3-4e27-bbd4-680f15fd43fe None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 24 05:58:24.786893 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.sql.core [None req-145053bf-9ce3-4e27-bbd4-680f15fd43fe None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 24 05:58:24.786893 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 24 05:58:24.786893 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [parameters: {'id': '47023ecf01434b48b2e948466751b419', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 24 05:58:24.786893 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81852) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 24 05:58:24.804548 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-145053bf-9ce3-4e27-bbd4-680f15fd43fe None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 47023ecf01434b48b2e948466751b419.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 47023ecf01434b48b2e948466751b419. Aug 24 05:58:24.804548 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:58:24.804548 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 24 05:58:24.804548 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 24 05:58:24.804548 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 24 05:58:24.804548 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 24 05:58:24.804548 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 24 05:58:24.804548 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application result = self._query(query) Aug 24 05:58:24.804548 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 24 05:58:24.804548 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application conn.query(q) Aug 24 05:58:24.804548 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 24 05:58:24.804548 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 24 05:58:24.804548 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 24 05:58:24.805415 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application result.read() Aug 24 05:58:24.805415 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 24 05:58:24.805415 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 24 05:58:24.805415 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 24 05:58:24.805415 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 24 05:58:24.805415 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 24 05:58:24.805415 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 24 05:58:24.805415 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 24 05:58:24.805415 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 24 05:58:24.805415 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 24 05:58:24.805415 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 24 05:58:24.805415 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 24 05:58:24.805415 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 24 05:58:24.805415 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:58:24.805415 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 24 05:58:24.805415 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 24 05:58:24.805415 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Aug 24 05:58:24.805415 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return project_ref.to_dict() Aug 24 05:58:24.806384 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 24 05:58:24.806384 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application next(self.gen) Aug 24 05:58:24.806384 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 24 05:58:24.806384 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application yield resource Aug 24 05:58:24.806384 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 24 05:58:24.806384 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application next(self.gen) Aug 24 05:58:24.806384 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 24 05:58:24.806384 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application self.session.rollback() Aug 24 05:58:24.806384 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 24 05:58:24.806384 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application self.force_reraise() Aug 24 05:58:24.806384 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 24 05:58:24.806384 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise self.value Aug 24 05:58:24.806384 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 24 05:58:24.806384 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 24 05:58:24.806384 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 24 05:58:24.806384 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application session.commit() Aug 24 05:58:24.806384 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 24 05:58:24.807280 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 24 05:58:24.807280 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 24 05:58:24.807280 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application self._prepare_impl() Aug 24 05:58:24.807280 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 24 05:58:24.807280 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application self.session.flush() Aug 24 05:58:24.807280 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 24 05:58:24.807280 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application self._flush(objects) Aug 24 05:58:24.807280 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 24 05:58:24.807280 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 24 05:58:24.807280 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 24 05:58:24.807280 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application compat.raise_( Aug 24 05:58:24.807280 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 24 05:58:24.807280 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception Aug 24 05:58:24.807280 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 24 05:58:24.807280 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application flush_context.execute() Aug 24 05:58:24.807280 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 24 05:58:24.807280 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rec.execute(self) Aug 24 05:58:24.808388 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 24 05:58:24.808388 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 24 05:58:24.808388 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 24 05:58:24.808388 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 24 05:58:24.808388 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 24 05:58:24.808388 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 24 05:58:24.808388 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 24 05:58:24.808388 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 24 05:58:24.808388 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 24 05:58:24.808388 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 24 05:58:24.808388 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 24 05:58:24.808388 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 24 05:58:24.808388 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 24 05:58:24.808388 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 24 05:58:24.808388 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 24 05:58:24.809247 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 24 05:58:24.809247 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 24 05:58:24.809247 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception Aug 24 05:58:24.809247 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 24 05:58:24.809247 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 24 05:58:24.809247 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 24 05:58:24.809247 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 24 05:58:24.809247 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 24 05:58:24.809247 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application result = self._query(query) Aug 24 05:58:24.809247 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 24 05:58:24.809247 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application conn.query(q) Aug 24 05:58:24.809247 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 24 05:58:24.809247 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 24 05:58:24.809247 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 24 05:58:24.809247 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application result.read() Aug 24 05:58:24.809247 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 24 05:58:24.809247 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 24 05:58:24.810119 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 24 05:58:24.810119 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 24 05:58:24.810119 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 24 05:58:24.810119 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 24 05:58:24.810119 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 24 05:58:24.810119 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 24 05:58:24.810119 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 24 05:58:24.810119 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 24 05:58:24.810119 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [parameters: {'id': '47023ecf01434b48b2e948466751b419', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 24 05:58:24.810119 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 24 05:58:24.810119 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 24 05:58:24.810119 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 05:58:24.810119 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 24 05:58:24.810119 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:58:24.810119 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Aug 24 05:58:24.811488 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Aug 24 05:58:24.811488 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 24 05:58:24.811488 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 24 05:58:24.811488 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Aug 24 05:58:24.811488 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 24 05:58:24.811488 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 05:58:24.811488 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 24 05:58:24.811488 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:58:24.811488 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:58:24.811488 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:58:24.811488 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:58:24.811488 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:58:24.811488 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:58:24.811488 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:58:24.811488 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:58:24.811488 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:58:24.811488 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:58:24.812437 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:58:24.812437 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Aug 24 05:58:24.812437 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Aug 24 05:58:24.812437 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:58:24.812437 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:58:24.812437 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Aug 24 05:58:24.812437 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.Conflict( Aug 24 05:58:24.812437 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 47023ecf01434b48b2e948466751b419. Aug 24 05:58:24.812437 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 24 05:58:24.812437 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 471/947] 10.222.0.62 () {60 vars in 1140 bytes} [Wed Aug 24 05:58:24 2022] POST /identity/v3/projects => generated 262 bytes in 216 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:58:24.817241 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce933c76-67e1-47b8-8ae1-ea2d2ee674dc None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:24.926203 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce933c76-67e1-47b8-8ae1-ea2d2ee674dc None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:24.947145 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:24.962871 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fc7437-79fa-42df-98ff-33576cae75aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:24.963710 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fc7437-79fa-42df-98ff-33576cae75aa None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:24.964141 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fc7437-79fa-42df-98ff-33576cae75aa None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:24.964609 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fc7437-79fa-42df-98ff-33576cae75aa None admin] PATH_INFO: `/v3/projects/service` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:24.971332 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79fc7437-79fa-42df-98ff-33576cae75aa None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:24.973060 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79fc7437-79fa-42df-98ff-33576cae75aa None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:24.987145 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application [None req-79fc7437-79fa-42df-98ff-33576cae75aa None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 05:58:24.987145 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:58:24.987145 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:58:24.987145 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:58:24.987145 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:58:24.987145 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:58:24.987145 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:58:24.987145 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:58:24.987145 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:58:24.987145 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:58:24.987145 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:58:24.987145 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:58:24.987145 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 05:58:24.987145 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 05:58:24.987145 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 05:58:24.988407 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 05:58:24.988407 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:58:24.988407 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:58:24.988407 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:58:24.988407 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:58:24.988407 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:58:24.988407 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:58:24.988407 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:58:24.988407 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application with Lock( Aug 24 05:58:24.988407 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:58:24.988407 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:58:24.988407 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:58:24.988407 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:58:24.988407 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:58:24.988407 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:58:24.988407 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:58:24.988407 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:58:24.989523 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 05:58:24.989523 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 05:58:24.989523 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 05:58:24.989523 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 05:58:24.989523 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 05:58:24.989523 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 05:58:24.989523 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 05:58:24.989523 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Aug 24 05:58:24.994697 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 477/948] 10.222.0.62 () {56 vars in 1111 bytes} [Wed Aug 24 05:58:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:58:24.999134 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145053bf-9ce3-4e27-bbd4-680f15fd43fe None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:25.099015 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:25.102707 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145053bf-9ce3-4e27-bbd4-680f15fd43fe None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:25.137773 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cebb299-7ab2-420f-8441-4129ba648945 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:25.138465 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cebb299-7ab2-420f-8441-4129ba648945 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:25.138741 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cebb299-7ab2-420f-8441-4129ba648945 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:25.139010 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cebb299-7ab2-420f-8441-4129ba648945 None admin] PATH_INFO: `/v3/projects` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:25.139693 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cebb299-7ab2-420f-8441-4129ba648945 None admin] RBAC: Adding query filter params (name=service) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:58:25.140043 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cebb299-7ab2-420f-8441-4129ba648945 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:25.141689 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cebb299-7ab2-420f-8441-4129ba648945 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:25.161770 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 472/949] 10.222.0.62 () {56 vars in 1112 bytes} [Wed Aug 24 05:58:24 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:58:25.190220 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfccaac2-6c0c-427d-80ad-a6582494b27b None None] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:25.190415 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfccaac2-6c0c-427d-80ad-a6582494b27b None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:25.190668 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfccaac2-6c0c-427d-80ad-a6582494b27b None None] PATH_INFO: `/` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:25.192199 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 478/950] 10.222.0.62 () {54 vars in 927 bytes} [Wed Aug 24 05:58:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:58:25.197887 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16eca5f-364e-4efa-86b3-09ee9c90b060 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:25.198171 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16eca5f-364e-4efa-86b3-09ee9c90b060 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:25.198493 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16eca5f-364e-4efa-86b3-09ee9c90b060 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:25.218795 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-a16eca5f-364e-4efa-86b3-09ee9c90b060 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:58:25.220439 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-a16eca5f-364e-4efa-86b3-09ee9c90b060 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:58:25.302303 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 473/951] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Aug 24 05:58:25 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 05:58:25.570442 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfccaac2-6c0c-427d-80ad-a6582494b27b None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:25.672959 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfccaac2-6c0c-427d-80ad-a6582494b27b None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:25.718444 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babc1d33-deff-4199-b8d9-61233be473ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:25.719121 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babc1d33-deff-4199-b8d9-61233be473ab None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:25.719401 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babc1d33-deff-4199-b8d9-61233be473ab None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:25.719665 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babc1d33-deff-4199-b8d9-61233be473ab None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:25.720239 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-babc1d33-deff-4199-b8d9-61233be473ab None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:25.721636 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-babc1d33-deff-4199-b8d9-61233be473ab None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:25.730558 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application [None req-babc1d33-deff-4199-b8d9-61233be473ab None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 05:58:25.730558 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:58:25.730558 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 05:58:25.730558 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 05:58:25.730558 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 05:58:25.730558 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 05:58:25.730558 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 05:58:25.730558 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 05:58:25.730558 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 05:58:25.730558 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Aug 24 05:58:25.730558 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 05:58:25.730558 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Aug 24 05:58:25.730558 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:58:25.730558 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:58:25.730558 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:58:25.730558 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:58:25.730558 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:58:25.731838 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:58:25.731838 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:58:25.731838 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:58:25.731838 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:58:25.731838 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:58:25.731838 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:58:25.731838 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 05:58:25.731838 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 05:58:25.731838 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 05:58:25.731838 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 05:58:25.731838 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:58:25.731838 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:58:25.731838 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:58:25.731838 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:58:25.731838 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:58:25.731838 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:58:25.733812 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:58:25.733812 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application with Lock( Aug 24 05:58:25.733812 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:58:25.733812 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:58:25.733812 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:58:25.733812 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:58:25.733812 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:58:25.733812 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:58:25.733812 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:58:25.733812 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:58:25.733812 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 05:58:25.733812 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 05:58:25.733812 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 05:58:25.733812 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Aug 24 05:58:25.733812 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 479/952] 10.222.0.62 () {56 vars in 1126 bytes} [Wed Aug 24 05:58:25 2022] GET /identity/v3/services/object-store => generated 93 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:58:25.738435 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16eca5f-364e-4efa-86b3-09ee9c90b060 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:25.832957 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:25.842372 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16eca5f-364e-4efa-86b3-09ee9c90b060 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:25.877077 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96eb56ec-0270-493c-bf49-ba1b569e6e2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:25.877887 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96eb56ec-0270-493c-bf49-ba1b569e6e2d None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:25.878606 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96eb56ec-0270-493c-bf49-ba1b569e6e2d None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:25.878878 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96eb56ec-0270-493c-bf49-ba1b569e6e2d None admin] PATH_INFO: `/v3/services` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:25.879501 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96eb56ec-0270-493c-bf49-ba1b569e6e2d None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:58:25.879799 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96eb56ec-0270-493c-bf49-ba1b569e6e2d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:25.881191 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96eb56ec-0270-493c-bf49-ba1b569e6e2d None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:25.886421 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 474/953] 10.222.0.62 () {56 vars in 1122 bytes} [Wed Aug 24 05:58:25 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:58:25.891542 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064891f7-298f-481c-be08-ae198df1ccee None None] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:25.892089 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064891f7-298f-481c-be08-ae198df1ccee None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:25.892089 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064891f7-298f-481c-be08-ae198df1ccee None None] PATH_INFO: `/` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:25.893694 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 480/954] 10.222.0.62 () {54 vars in 927 bytes} [Wed Aug 24 05:58:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:58:25.893878 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96eb56ec-0270-493c-bf49-ba1b569e6e2d None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:25.898956 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37468f59-0970-411b-a6df-322b3ea0c191 None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:25.899239 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37468f59-0970-411b-a6df-322b3ea0c191 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:25.899516 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37468f59-0970-411b-a6df-322b3ea0c191 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:25.920759 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-37468f59-0970-411b-a6df-322b3ea0c191 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:58:25.922525 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-37468f59-0970-411b-a6df-322b3ea0c191 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:58:26.000890 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:26.001808 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96eb56ec-0270-493c-bf49-ba1b569e6e2d None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:26.011145 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 481/955] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Aug 24 05:58:25 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 05:58:26.046980 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af911b1a-f39b-4b67-acb7-2cfb4f090fa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:26.047932 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af911b1a-f39b-4b67-acb7-2cfb4f090fa3 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:26.048517 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af911b1a-f39b-4b67-acb7-2cfb4f090fa3 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:26.049003 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af911b1a-f39b-4b67-acb7-2cfb4f090fa3 None admin] PATH_INFO: `/v3/services` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:26.049883 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af911b1a-f39b-4b67-acb7-2cfb4f090fa3 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:58:26.050408 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af911b1a-f39b-4b67-acb7-2cfb4f090fa3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:26.052026 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af911b1a-f39b-4b67-acb7-2cfb4f090fa3 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:26.057702 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 475/956] 10.222.0.62 () {56 vars in 1122 bytes} [Wed Aug 24 05:58:25 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:58:26.063056 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37468f59-0970-411b-a6df-322b3ea0c191 None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:26.165999 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37468f59-0970-411b-a6df-322b3ea0c191 None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:26.202321 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbdd4ce-9173-4042-a17b-88bb725c35cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:26.203369 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbdd4ce-9173-4042-a17b-88bb725c35cd None admin] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:26.203864 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbdd4ce-9173-4042-a17b-88bb725c35cd None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:26.204341 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbdd4ce-9173-4042-a17b-88bb725c35cd None admin] PATH_INFO: `/v3/endpoints` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:26.205227 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbdd4ce-9173-4042-a17b-88bb725c35cd None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:26.206783 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbdd4ce-9173-4042-a17b-88bb725c35cd None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:26.283831 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af911b1a-f39b-4b67-acb7-2cfb4f090fa3 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:26.311267 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:26.320063 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 482/957] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Aug 24 05:58:26 2022] POST /identity/v3/endpoints => generated 349 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:58:26.354746 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59de85fe-d714-4ae1-a096-f1451fb1aaa6 None None] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:26.355086 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59de85fe-d714-4ae1-a096-f1451fb1aaa6 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:26.355297 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59de85fe-d714-4ae1-a096-f1451fb1aaa6 None None] PATH_INFO: `/` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:26.358120 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 483/958] 10.222.0.62 () {54 vars in 927 bytes} [Wed Aug 24 05:58:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 05:58:26.362757 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f76dd2-3b5f-4255-99d3-29dfc1536de0 None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:26.363038 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f76dd2-3b5f-4255-99d3-29dfc1536de0 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:26.365831 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f76dd2-3b5f-4255-99d3-29dfc1536de0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:26.383204 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-f0f76dd2-3b5f-4255-99d3-29dfc1536de0 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:58:26.384593 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-f0f76dd2-3b5f-4255-99d3-29dfc1536de0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:58:26.412962 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:26.417825 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af911b1a-f39b-4b67-acb7-2cfb4f090fa3 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:26.469379 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951f06f2-4d5f-4fe9-9d83-8d23bb9630ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:26.470009 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951f06f2-4d5f-4fe9-9d83-8d23bb9630ed None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:26.470281 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951f06f2-4d5f-4fe9-9d83-8d23bb9630ed None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:26.470546 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951f06f2-4d5f-4fe9-9d83-8d23bb9630ed None admin] PATH_INFO: `/v3/services/compute` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:26.471112 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951f06f2-4d5f-4fe9-9d83-8d23bb9630ed None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:26.472637 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951f06f2-4d5f-4fe9-9d83-8d23bb9630ed None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:26.474790 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 484/959] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Aug 24 05:58:26 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 05:58:26.484405 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-951f06f2-4d5f-4fe9-9d83-8d23bb9630ed None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 05:58:26.484405 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:58:26.484405 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 05:58:26.484405 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 05:58:26.484405 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 05:58:26.484405 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 05:58:26.484405 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 05:58:26.484405 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 05:58:26.484405 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 05:58:26.484405 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 24 05:58:26.484405 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 05:58:26.484405 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 24 05:58:26.484405 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:58:26.484405 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:58:26.484405 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:58:26.484405 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:58:26.484405 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:58:26.485555 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:58:26.485555 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:58:26.485555 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:58:26.485555 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:58:26.485555 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:58:26.485555 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:58:26.485555 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 05:58:26.485555 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 05:58:26.485555 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 05:58:26.485555 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 05:58:26.485555 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:58:26.485555 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:58:26.485555 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:58:26.485555 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:58:26.485555 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:58:26.485555 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:58:26.486418 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:58:26.486418 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 24 05:58:26.486418 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:58:26.486418 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:58:26.486418 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:58:26.486418 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:58:26.486418 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:58:26.486418 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:58:26.486418 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:58:26.486418 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:58:26.486418 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 05:58:26.486418 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 05:58:26.486418 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 05:58:26.486418 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 24 05:58:26.486418 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 476/960] 10.222.0.62 () {56 vars in 1111 bytes} [Wed Aug 24 05:58:26 2022] GET /identity/v3/services/compute => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 05:58:26.493730 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f76dd2-3b5f-4255-99d3-29dfc1536de0 None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:26.599501 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f76dd2-3b5f-4255-99d3-29dfc1536de0 None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:26.634632 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517acfa4-3eca-4711-8917-71af5d7d53b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:26.635608 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517acfa4-3eca-4711-8917-71af5d7d53b6 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:26.636114 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517acfa4-3eca-4711-8917-71af5d7d53b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:26.636623 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517acfa4-3eca-4711-8917-71af5d7d53b6 None admin] PATH_INFO: `/v3/services` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:26.637464 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517acfa4-3eca-4711-8917-71af5d7d53b6 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:58:26.637981 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517acfa4-3eca-4711-8917-71af5d7d53b6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:26.640069 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517acfa4-3eca-4711-8917-71af5d7d53b6 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:26.645609 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 485/961] 10.222.0.62 () {56 vars in 1112 bytes} [Wed Aug 24 05:58:26 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:58:26.651419 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951f06f2-4d5f-4fe9-9d83-8d23bb9630ed None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:26.745329 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:26.767004 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517acfa4-3eca-4711-8917-71af5d7d53b6 None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:26.772238 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951f06f2-4d5f-4fe9-9d83-8d23bb9630ed None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:26.801069 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:26.812373 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802c58bb-41fe-4462-a916-fc4240e7469a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:26.813033 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802c58bb-41fe-4462-a916-fc4240e7469a None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:26.813309 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802c58bb-41fe-4462-a916-fc4240e7469a None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:26.813616 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802c58bb-41fe-4462-a916-fc4240e7469a None admin] PATH_INFO: `/v3/services` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:26.814206 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802c58bb-41fe-4462-a916-fc4240e7469a None admin] RBAC: Adding query filter params (type=compute) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:58:26.814506 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802c58bb-41fe-4462-a916-fc4240e7469a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:26.816759 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802c58bb-41fe-4462-a916-fc4240e7469a None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:26.821237 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 477/962] 10.222.0.62 () {56 vars in 1112 bytes} [Wed Aug 24 05:58:26 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:58:26.898797 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517acfa4-3eca-4711-8917-71af5d7d53b6 None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:26.937041 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc71f7e8-588f-4883-88bd-5ca56cb06758 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:26.937698 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc71f7e8-588f-4883-88bd-5ca56cb06758 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:26.938029 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc71f7e8-588f-4883-88bd-5ca56cb06758 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:26.938317 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc71f7e8-588f-4883-88bd-5ca56cb06758 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:26.950022 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc71f7e8-588f-4883-88bd-5ca56cb06758 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:58:26.955411 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application [None req-bc71f7e8-588f-4883-88bd-5ca56cb06758 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 05:58:26.955411 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:58:26.955411 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:58:26.955411 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:58:26.955411 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:58:26.955411 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:58:26.955411 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:58:26.955411 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:58:26.955411 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:58:26.955411 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:58:26.955411 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:58:26.955411 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:58:26.955411 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:58:26.955411 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:58:26.955411 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:58:26.955411 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:58:26.956415 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:58:26.956415 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:58:26.956415 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:58:26.956415 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:58:26.956415 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:58:26.956415 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:58:26.956415 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:58:26.956415 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:58:26.956415 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:58:26.956415 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:58:26.956415 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:58:26.956415 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application with Lock( Aug 24 05:58:26.956415 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:58:26.956415 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:58:26.956415 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:58:26.956415 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:58:26.957382 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:58:26.957382 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:58:26.957382 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:58:26.957382 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:58:26.957382 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:58:26.957382 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:58:26.957382 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:58:26.957382 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:58:26.957382 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:58:26.957382 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:58:26.957382 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 05:58:26.957382 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Aug 24 05:58:26.957382 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 486/963] 10.222.0.62 () {56 vars in 1120 bytes} [Wed Aug 24 05:58:26 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 05:58:26.963011 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802c58bb-41fe-4462-a916-fc4240e7469a None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:27.069576 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802c58bb-41fe-4462-a916-fc4240e7469a None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:27.103750 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124a9935-642d-40a4-8dbc-7cbb197538a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:27.108371 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124a9935-642d-40a4-8dbc-7cbb197538a9 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:27.108698 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124a9935-642d-40a4-8dbc-7cbb197538a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:27.108974 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124a9935-642d-40a4-8dbc-7cbb197538a9 None admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:27.109667 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124a9935-642d-40a4-8dbc-7cbb197538a9 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:58:27.109969 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124a9935-642d-40a4-8dbc-7cbb197538a9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:27.111347 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124a9935-642d-40a4-8dbc-7cbb197538a9 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:27.120316 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 478/964] 10.222.0.62 () {56 vars in 1115 bytes} [Wed Aug 24 05:58:26 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:58:27.126602 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc71f7e8-588f-4883-88bd-5ca56cb06758 None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:27.145674 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2561637c-1720-4ccb-9bc1-cac0b24da69c None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:27.146003 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2561637c-1720-4ccb-9bc1-cac0b24da69c None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:27.146223 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2561637c-1720-4ccb-9bc1-cac0b24da69c None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:27.147689 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 479/965] 10.222.0.62 () {54 vars in 927 bytes} [Wed Aug 24 05:58:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 05:58:27.153007 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6a0765-a61b-445d-8965-f9012ddf9bdf None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:27.153290 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6a0765-a61b-445d-8965-f9012ddf9bdf None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:27.153559 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6a0765-a61b-445d-8965-f9012ddf9bdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:27.160415 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:27.175815 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-7b6a0765-a61b-445d-8965-f9012ddf9bdf None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:58:27.177042 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-7b6a0765-a61b-445d-8965-f9012ddf9bdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:58:27.227160 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:27.244566 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc71f7e8-588f-4883-88bd-5ca56cb06758 None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:27.270433 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 480/966] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Aug 24 05:58:27 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 05:58:27.287801 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f525d15-35cf-4416-9c32-f6be97d9bd95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:27.288438 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f525d15-35cf-4416-9c32-f6be97d9bd95 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:27.288817 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f525d15-35cf-4416-9c32-f6be97d9bd95 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:27.289094 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f525d15-35cf-4416-9c32-f6be97d9bd95 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:27.300679 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f525d15-35cf-4416-9c32-f6be97d9bd95 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:27.302206 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f525d15-35cf-4416-9c32-f6be97d9bd95 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:27.317990 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application [None req-5f525d15-35cf-4416-9c32-f6be97d9bd95 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:58:27.317990 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:58:27.317990 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:58:27.317990 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:58:27.317990 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:58:27.317990 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:58:27.317990 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:58:27.317990 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:58:27.317990 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:58:27.317990 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:58:27.317990 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:58:27.317990 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:58:27.317990 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 05:58:27.317990 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 05:58:27.317990 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 05:58:27.317990 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 05:58:27.318949 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:58:27.318949 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:58:27.318949 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:58:27.318949 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:58:27.318949 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:58:27.318949 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:58:27.318949 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:58:27.318949 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application with Lock( Aug 24 05:58:27.318949 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:58:27.318949 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:58:27.318949 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:58:27.318949 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:58:27.318949 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:58:27.318949 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:58:27.318949 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:58:27.318949 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:58:27.318949 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 05:58:27.319832 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 05:58:27.319832 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:58:27.319832 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Aug 24 05:58:27.319832 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 487/967] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Aug 24 05:58:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:58:27.325775 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6a0765-a61b-445d-8965-f9012ddf9bdf None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:27.428883 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6a0765-a61b-445d-8965-f9012ddf9bdf None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:27.462243 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:27.467630 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cec8cf-8829-41fb-b306-77a120e3dc44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:27.468242 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cec8cf-8829-41fb-b306-77a120e3dc44 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:27.468540 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cec8cf-8829-41fb-b306-77a120e3dc44 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:27.468820 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cec8cf-8829-41fb-b306-77a120e3dc44 None admin] PATH_INFO: `/v3/domains` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:27.469458 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23cec8cf-8829-41fb-b306-77a120e3dc44 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:58:27.469755 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23cec8cf-8829-41fb-b306-77a120e3dc44 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:27.471154 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23cec8cf-8829-41fb-b306-77a120e3dc44 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:27.483648 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 481/968] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Aug 24 05:58:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:58:27.488874 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f525d15-35cf-4416-9c32-f6be97d9bd95 None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:27.540624 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cec8cf-8829-41fb-b306-77a120e3dc44 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:27.599193 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f525d15-35cf-4416-9c32-f6be97d9bd95 None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:27.637726 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11118224-fbab-4487-84b5-a0775f18bb1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:27.638528 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11118224-fbab-4487-84b5-a0775f18bb1a None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:27.638813 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11118224-fbab-4487-84b5-a0775f18bb1a None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:27.639084 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11118224-fbab-4487-84b5-a0775f18bb1a None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:27.643193 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:27.645219 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cec8cf-8829-41fb-b306-77a120e3dc44 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:27.648732 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11118224-fbab-4487-84b5-a0775f18bb1a None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:27.650182 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11118224-fbab-4487-84b5-a0775f18bb1a None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:27.662541 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application [None req-11118224-fbab-4487-84b5-a0775f18bb1a None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 05:58:27.662541 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:58:27.662541 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:58:27.662541 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:58:27.662541 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:58:27.662541 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:58:27.662541 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:58:27.662541 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:58:27.662541 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:58:27.662541 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:58:27.662541 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:58:27.662541 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:58:27.662541 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 05:58:27.662541 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 05:58:27.662541 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 05:58:27.662541 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 05:58:27.663705 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:58:27.663705 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:58:27.663705 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 05:58:27.663705 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 05:58:27.663705 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 05:58:27.663705 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 05:58:27.663705 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:58:27.663705 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:58:27.663705 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:58:27.663705 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:58:27.663705 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:58:27.663705 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application with Lock( Aug 24 05:58:27.663705 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:58:27.663705 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:58:27.663705 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:58:27.663705 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:58:27.663705 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:58:27.663705 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:58:27.664940 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:58:27.664940 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:58:27.664940 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 05:58:27.664940 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 05:58:27.664940 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 05:58:27.664940 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 05:58:27.664940 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 05:58:27.664940 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 05:58:27.664940 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 05:58:27.664940 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Aug 24 05:58:27.668484 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 488/969] 10.222.0.62 () {56 vars in 1117 bytes} [Wed Aug 24 05:58:27 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:58:27.670640 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11118224-fbab-4487-84b5-a0775f18bb1a None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:27.692586 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63648044-d8ec-4617-9584-fc988c8a2d5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:27.692586 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63648044-d8ec-4617-9584-fc988c8a2d5a None admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:27.692586 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63648044-d8ec-4617-9584-fc988c8a2d5a None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:27.692586 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63648044-d8ec-4617-9584-fc988c8a2d5a None admin] PATH_INFO: `/v3/users` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:27.692972 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63648044-d8ec-4617-9584-fc988c8a2d5a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:27.695028 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63648044-d8ec-4617-9584-fc988c8a2d5a None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:27.746804 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.sql.core [None req-63648044-d8ec-4617-9584-fc988c8a2d5a None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 24 05:58:27.746804 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 24 05:58:27.746804 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [parameters: {'user_id': '8d2dc62fb30f406cad2875e7da06d6ec', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 24 05:58:27.746804 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81852) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 24 05:58:27.753933 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-63648044-d8ec-4617-9584-fc988c8a2d5a None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 24 05:58:27.753933 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:58:27.753933 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 24 05:58:27.753933 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 24 05:58:27.753933 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 24 05:58:27.753933 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 24 05:58:27.753933 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 24 05:58:27.753933 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application result = self._query(query) Aug 24 05:58:27.753933 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 24 05:58:27.753933 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application conn.query(q) Aug 24 05:58:27.753933 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 24 05:58:27.753933 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 24 05:58:27.753933 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 24 05:58:27.753933 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application result.read() Aug 24 05:58:27.753933 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 24 05:58:27.754819 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 24 05:58:27.754819 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 24 05:58:27.754819 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 24 05:58:27.754819 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 24 05:58:27.754819 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 24 05:58:27.754819 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 24 05:58:27.754819 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 24 05:58:27.754819 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 24 05:58:27.754819 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 24 05:58:27.754819 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 24 05:58:27.754819 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 24 05:58:27.754819 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:58:27.754819 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 24 05:58:27.754819 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 24 05:58:27.754819 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Aug 24 05:58:27.754819 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Aug 24 05:58:27.754819 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 24 05:58:27.754819 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application next(self.gen) Aug 24 05:58:27.755806 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 24 05:58:27.755806 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application yield resource Aug 24 05:58:27.755806 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 24 05:58:27.755806 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application next(self.gen) Aug 24 05:58:27.755806 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 24 05:58:27.755806 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application self.session.rollback() Aug 24 05:58:27.755806 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 24 05:58:27.755806 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application self.force_reraise() Aug 24 05:58:27.755806 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 24 05:58:27.755806 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise self.value Aug 24 05:58:27.755806 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 24 05:58:27.755806 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 24 05:58:27.755806 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 24 05:58:27.755806 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application session.commit() Aug 24 05:58:27.755806 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 24 05:58:27.755806 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 24 05:58:27.755806 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 24 05:58:27.756745 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application self._prepare_impl() Aug 24 05:58:27.756745 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 24 05:58:27.756745 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application self.session.flush() Aug 24 05:58:27.756745 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 24 05:58:27.756745 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application self._flush(objects) Aug 24 05:58:27.756745 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 24 05:58:27.756745 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 24 05:58:27.756745 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 24 05:58:27.756745 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application compat.raise_( Aug 24 05:58:27.756745 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 24 05:58:27.756745 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception Aug 24 05:58:27.756745 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 24 05:58:27.756745 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application flush_context.execute() Aug 24 05:58:27.756745 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 24 05:58:27.756745 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rec.execute(self) Aug 24 05:58:27.756745 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 24 05:58:27.756745 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 24 05:58:27.757938 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 24 05:58:27.757938 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 24 05:58:27.757938 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Aug 24 05:58:27.757938 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application result = connection._execute_20( Aug 24 05:58:27.757938 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 24 05:58:27.757938 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 24 05:58:27.757938 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 24 05:58:27.757938 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 24 05:58:27.757938 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 24 05:58:27.757938 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 24 05:58:27.757938 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 24 05:58:27.757938 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 24 05:58:27.757938 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 24 05:58:27.757938 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 24 05:58:27.757938 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 24 05:58:27.757938 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception Aug 24 05:58:27.758826 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 24 05:58:27.758826 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 24 05:58:27.758826 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 24 05:58:27.758826 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 24 05:58:27.758826 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 24 05:58:27.758826 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application result = self._query(query) Aug 24 05:58:27.758826 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 24 05:58:27.758826 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application conn.query(q) Aug 24 05:58:27.758826 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 24 05:58:27.758826 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 24 05:58:27.758826 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 24 05:58:27.758826 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application result.read() Aug 24 05:58:27.758826 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 24 05:58:27.758826 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 24 05:58:27.758826 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 24 05:58:27.758826 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 24 05:58:27.761923 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 24 05:58:27.761923 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 24 05:58:27.761923 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 24 05:58:27.761923 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 24 05:58:27.761923 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 24 05:58:27.761923 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 24 05:58:27.761923 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [parameters: {'user_id': '8d2dc62fb30f406cad2875e7da06d6ec', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 24 05:58:27.761923 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 24 05:58:27.761923 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 24 05:58:27.761923 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 05:58:27.761923 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 24 05:58:27.761923 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:58:27.761923 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:58:27.761923 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:58:27.761923 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:58:27.763874 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:58:27.763874 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:58:27.763874 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:58:27.763874 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:58:27.763874 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:58:27.763874 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:58:27.763874 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:58:27.763874 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Aug 24 05:58:27.763874 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Aug 24 05:58:27.763874 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:58:27.763874 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:58:27.763874 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 05:58:27.763874 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 05:58:27.763874 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 05:58:27.763874 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 05:58:27.763874 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Aug 24 05:58:27.763874 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Aug 24 05:58:27.766059 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Aug 24 05:58:27.766059 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Aug 24 05:58:27.766059 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 24 05:58:27.766059 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 24 05:58:27.766059 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 24 05:58:27.766059 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 24 05:58:27.766059 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 482/970] 10.222.0.62 () {60 vars in 1131 bytes} [Wed Aug 24 05:58:27 2022] POST /identity/v3/users => generated 160 bytes in 215 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Aug 24 05:58:27.766059 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63648044-d8ec-4617-9584-fc988c8a2d5a None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:27.805347 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11118224-fbab-4487-84b5-a0775f18bb1a None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:27.823220 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:27.853022 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4cb529-5912-4dbd-8ddf-ce4796553f4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:27.854198 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4cb529-5912-4dbd-8ddf-ce4796553f4c None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:27.854662 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4cb529-5912-4dbd-8ddf-ce4796553f4c None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:27.855123 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4cb529-5912-4dbd-8ddf-ce4796553f4c None admin] PATH_INFO: `/v3/users` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:27.856161 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4cb529-5912-4dbd-8ddf-ce4796553f4c None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:58:27.857008 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4cb529-5912-4dbd-8ddf-ce4796553f4c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:27.859242 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4cb529-5912-4dbd-8ddf-ce4796553f4c None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:27.878146 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63648044-d8ec-4617-9584-fc988c8a2d5a None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:27.884956 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 489/971] 10.222.0.62 () {56 vars in 1149 bytes} [Wed Aug 24 05:58:27 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:58:27.886847 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4cb529-5912-4dbd-8ddf-ce4796553f4c None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:27.922416 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a05fd4-b0e8-4547-a793-2e1105afebbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:27.923673 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a05fd4-b0e8-4547-a793-2e1105afebbe None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:27.924058 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a05fd4-b0e8-4547-a793-2e1105afebbe None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:27.924436 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a05fd4-b0e8-4547-a793-2e1105afebbe None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:27.934881 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a05fd4-b0e8-4547-a793-2e1105afebbe None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:27.937753 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a05fd4-b0e8-4547-a793-2e1105afebbe None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:27.953610 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-36a05fd4-b0e8-4547-a793-2e1105afebbe None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 05:58:27.953610 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:58:27.953610 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:58:27.953610 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:58:27.953610 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:58:27.953610 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:58:27.953610 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:58:27.953610 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:58:27.953610 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:58:27.953610 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:58:27.953610 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:58:27.953610 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:58:27.953610 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 05:58:27.953610 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 05:58:27.953610 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 05:58:27.953610 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 05:58:27.954786 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:58:27.954786 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:58:27.954786 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 05:58:27.954786 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 05:58:27.954786 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 05:58:27.954786 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 05:58:27.954786 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:58:27.954786 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:58:27.954786 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:58:27.954786 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:58:27.954786 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:58:27.954786 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 24 05:58:27.954786 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:58:27.954786 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:58:27.954786 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:58:27.954786 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:58:27.954786 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:58:27.954786 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:58:27.955937 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:58:27.955937 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:58:27.955937 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 05:58:27.955937 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 05:58:27.955937 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 05:58:27.955937 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 05:58:27.955937 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 05:58:27.955937 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 05:58:27.955937 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 05:58:27.955937 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 24 05:58:27.958115 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 483/972] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Aug 24 05:58:27 2022] GET /identity/v3/users/cinder => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:58:27.960062 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:27.963671 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a05fd4-b0e8-4547-a793-2e1105afebbe None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:28.006341 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4cb529-5912-4dbd-8ddf-ce4796553f4c None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:28.044072 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e0f88b-914f-418f-9277-55c0f15ee5e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:28.045717 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e0f88b-914f-418f-9277-55c0f15ee5e9 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:28.046161 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e0f88b-914f-418f-9277-55c0f15ee5e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:28.046577 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e0f88b-914f-418f-9277-55c0f15ee5e9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:28.076994 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a05fd4-b0e8-4547-a793-2e1105afebbe None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:28.092310 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:28.115357 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e591b1e0-0796-4783-9ff5-0f1615741c36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:28.116384 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e591b1e0-0796-4783-9ff5-0f1615741c36 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:28.116923 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e591b1e0-0796-4783-9ff5-0f1615741c36 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:28.117465 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e591b1e0-0796-4783-9ff5-0f1615741c36 None admin] PATH_INFO: `/v3/users` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:28.118270 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e0f88b-914f-418f-9277-55c0f15ee5e9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:28.118604 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e591b1e0-0796-4783-9ff5-0f1615741c36 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:58:28.119137 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e591b1e0-0796-4783-9ff5-0f1615741c36 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:28.119801 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e0f88b-914f-418f-9277-55c0f15ee5e9 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:28.121038 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e591b1e0-0796-4783-9ff5-0f1615741c36 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:28.163524 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 484/973] 10.222.0.62 () {56 vars in 1101 bytes} [Wed Aug 24 05:58:27 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:58:28.178424 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:28.230061 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 490/974] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Aug 24 05:58:27 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 24 05:58:28.236292 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e591b1e0-0796-4783-9ff5-0f1615741c36 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:28.348762 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e591b1e0-0796-4783-9ff5-0f1615741c36 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:28.385472 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e23c43-d0ea-4614-ae7e-535ae686e067 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:28.386639 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e23c43-d0ea-4614-ae7e-535ae686e067 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:28.387121 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e23c43-d0ea-4614-ae7e-535ae686e067 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:28.387611 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e23c43-d0ea-4614-ae7e-535ae686e067 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:28.399489 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e23c43-d0ea-4614-ae7e-535ae686e067 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:28.401218 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e23c43-d0ea-4614-ae7e-535ae686e067 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:28.405184 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49937118-4769-4bb1-b6cf-3326bd800a86 None None] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:28.405472 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49937118-4769-4bb1-b6cf-3326bd800a86 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:28.405752 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49937118-4769-4bb1-b6cf-3326bd800a86 None None] PATH_INFO: `/` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:28.407101 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 491/975] 10.222.0.62 () {54 vars in 927 bytes} [Wed Aug 24 05:58:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:58:28.412333 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1977e2-7914-4fa5-8ca0-972900920cf8 None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:28.412648 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1977e2-7914-4fa5-8ca0-972900920cf8 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:28.412924 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1977e2-7914-4fa5-8ca0-972900920cf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:28.418789 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-93e23c43-d0ea-4614-ae7e-535ae686e067 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:58:28.418789 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:58:28.418789 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:58:28.418789 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:58:28.418789 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:58:28.418789 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:58:28.418789 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:58:28.418789 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:58:28.418789 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:58:28.418789 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:58:28.418789 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:58:28.418789 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:58:28.418789 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 05:58:28.418789 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 05:58:28.418789 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 05:58:28.418789 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 05:58:28.420274 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:58:28.420274 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:58:28.420274 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:58:28.420274 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:58:28.420274 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:58:28.420274 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:58:28.420274 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:58:28.420274 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 24 05:58:28.420274 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:58:28.420274 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:58:28.420274 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:58:28.420274 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:58:28.420274 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:58:28.420274 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:58:28.420274 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:58:28.420274 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:58:28.420274 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 05:58:28.421403 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 05:58:28.421403 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:58:28.421403 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 24 05:58:28.424336 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 485/976] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Aug 24 05:58:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:58:28.428292 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e23c43-d0ea-4614-ae7e-535ae686e067 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:28.434982 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-fe1977e2-7914-4fa5-8ca0-972900920cf8 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:58:28.437587 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-fe1977e2-7914-4fa5-8ca0-972900920cf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:58:28.459047 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:28.522448 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 492/977] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Aug 24 05:58:28 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 05:58:28.535764 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e23c43-d0ea-4614-ae7e-535ae686e067 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:28.569776 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb43426b-d24b-4e22-939d-12add912eb39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:28.570377 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb43426b-d24b-4e22-939d-12add912eb39 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:28.570654 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb43426b-d24b-4e22-939d-12add912eb39 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:28.570921 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb43426b-d24b-4e22-939d-12add912eb39 None admin] PATH_INFO: `/v3/domains` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:28.571540 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb43426b-d24b-4e22-939d-12add912eb39 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:58:28.571840 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb43426b-d24b-4e22-939d-12add912eb39 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:28.573222 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb43426b-d24b-4e22-939d-12add912eb39 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:28.589770 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 486/978] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Aug 24 05:58:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:58:28.591582 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1977e2-7914-4fa5-8ca0-972900920cf8 None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:28.674526 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:28.695650 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1977e2-7914-4fa5-8ca0-972900920cf8 None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:28.732725 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564cc2e7-7377-49f0-97c3-485fa4bab1ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:28.733350 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564cc2e7-7377-49f0-97c3-485fa4bab1ec None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:28.733630 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564cc2e7-7377-49f0-97c3-485fa4bab1ec None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:28.733902 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564cc2e7-7377-49f0-97c3-485fa4bab1ec None admin] PATH_INFO: `/v3/projects/service` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:28.740626 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564cc2e7-7377-49f0-97c3-485fa4bab1ec None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:28.742061 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564cc2e7-7377-49f0-97c3-485fa4bab1ec None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:28.753523 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application [None req-564cc2e7-7377-49f0-97c3-485fa4bab1ec None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 05:58:28.753523 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:58:28.753523 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:58:28.753523 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:58:28.753523 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:58:28.753523 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:58:28.753523 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:58:28.753523 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:58:28.753523 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:58:28.753523 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:58:28.753523 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:58:28.753523 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:58:28.753523 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 05:58:28.753523 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 05:58:28.753523 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 05:58:28.754395 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 05:58:28.754395 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:58:28.754395 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:58:28.754395 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:58:28.754395 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:58:28.754395 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:58:28.754395 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:58:28.754395 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:58:28.754395 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application with Lock( Aug 24 05:58:28.754395 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:58:28.754395 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:58:28.754395 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:58:28.754395 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:58:28.754395 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:58:28.754395 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:58:28.754395 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:58:28.754395 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:58:28.760199 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 05:58:28.760199 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 05:58:28.760199 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 05:58:28.760199 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 05:58:28.760199 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 05:58:28.760199 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 05:58:28.760199 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 05:58:28.760199 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Aug 24 05:58:28.760199 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 493/979] 10.222.0.62 () {56 vars in 1111 bytes} [Wed Aug 24 05:58:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 05:58:28.762219 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb43426b-d24b-4e22-939d-12add912eb39 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:28.790200 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564cc2e7-7377-49f0-97c3-485fa4bab1ec None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:28.851854 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:28.868752 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb43426b-d24b-4e22-939d-12add912eb39 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:28.901578 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564cc2e7-7377-49f0-97c3-485fa4bab1ec None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:28.904252 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b86baa-ff57-4b7a-9469-ef7624d3b417 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:28.904882 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b86baa-ff57-4b7a-9469-ef7624d3b417 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:28.905156 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b86baa-ff57-4b7a-9469-ef7624d3b417 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:28.905423 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b86baa-ff57-4b7a-9469-ef7624d3b417 None admin] PATH_INFO: `/v3/projects` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:28.906121 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b86baa-ff57-4b7a-9469-ef7624d3b417 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:58:28.906420 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b86baa-ff57-4b7a-9469-ef7624d3b417 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:28.907826 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b86baa-ff57-4b7a-9469-ef7624d3b417 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:28.919200 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 487/980] 10.222.0.62 () {56 vars in 1148 bytes} [Wed Aug 24 05:58:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:58:28.921379 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d7659f-2449-47f4-b62d-048695288011 None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:28.921652 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d7659f-2449-47f4-b62d-048695288011 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:28.921920 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d7659f-2449-47f4-b62d-048695288011 None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:28.923193 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 488/981] 10.222.0.62 () {54 vars in 927 bytes} [Wed Aug 24 05:58:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:58:28.925860 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d7659f-2449-47f4-b62d-048695288011 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:28.939329 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1d78e2-e0a8-4a5a-b7f1-3a1cfc865cdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:28.939955 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1d78e2-e0a8-4a5a-b7f1-3a1cfc865cdb None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:28.940230 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1d78e2-e0a8-4a5a-b7f1-3a1cfc865cdb None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:28.940524 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1d78e2-e0a8-4a5a-b7f1-3a1cfc865cdb None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:28.941104 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1d78e2-e0a8-4a5a-b7f1-3a1cfc865cdb None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:28.942461 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1d78e2-e0a8-4a5a-b7f1-3a1cfc865cdb None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:28.950885 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application [None req-dc1d78e2-e0a8-4a5a-b7f1-3a1cfc865cdb None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 05:58:28.950885 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:58:28.950885 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 05:58:28.950885 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 05:58:28.950885 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 05:58:28.950885 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 05:58:28.950885 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 05:58:28.950885 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 05:58:28.950885 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 05:58:28.950885 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Aug 24 05:58:28.950885 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 05:58:28.950885 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Aug 24 05:58:28.950885 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:58:28.950885 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:58:28.950885 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:58:28.950885 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:58:28.950885 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:58:28.952077 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:58:28.952077 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:58:28.952077 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:58:28.952077 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:58:28.952077 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:58:28.952077 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:58:28.952077 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 05:58:28.952077 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 05:58:28.952077 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 05:58:28.952077 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 05:58:28.952077 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:58:28.952077 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:58:28.952077 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:58:28.952077 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:58:28.952077 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:58:28.952077 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:58:28.953067 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:58:28.953067 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application with Lock( Aug 24 05:58:28.953067 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:58:28.953067 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:58:28.953067 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:58:28.953067 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:58:28.953067 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:58:28.953067 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:58:28.953067 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:58:28.953067 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:58:28.953067 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 05:58:28.953067 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 05:58:28.953067 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 05:58:28.953067 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Aug 24 05:58:28.953067 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 494/982] 10.222.0.62 () {56 vars in 1126 bytes} [Wed Aug 24 05:58:28 2022] GET /identity/v3/services/object-store => generated 93 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 05:58:28.953867 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e9ca25-bc24-420f-b3a2-bd0b6ea42092 None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:28.953867 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e9ca25-bc24-420f-b3a2-bd0b6ea42092 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:28.953998 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e9ca25-bc24-420f-b3a2-bd0b6ea42092 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:28.970721 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-00e9ca25-bc24-420f-b3a2-bd0b6ea42092 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:58:28.971913 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-00e9ca25-bc24-420f-b3a2-bd0b6ea42092 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:58:28.973875 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:29.029087 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d7659f-2449-47f4-b62d-048695288011 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:29.039949 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 495/983] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Aug 24 05:58:28 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 05:58:29.040805 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e9ca25-bc24-420f-b3a2-bd0b6ea42092 None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:29.062190 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0eabaf3-bce6-412b-bcdc-8437c0008466 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:29.062792 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0eabaf3-bce6-412b-bcdc-8437c0008466 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:29.063069 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0eabaf3-bce6-412b-bcdc-8437c0008466 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:29.063338 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0eabaf3-bce6-412b-bcdc-8437c0008466 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:29.064046 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0eabaf3-bce6-412b-bcdc-8437c0008466 None admin] RBAC: Adding query filter params (role.id=2499f7e98f3e498ab329637f00bb2da4, scope.project.id=181f4762037e454489d9552cc603179c, user.id=b82726a12436435cb931b986ef884c2e) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:58:29.064356 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0eabaf3-bce6-412b-bcdc-8437c0008466 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:29.065800 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0eabaf3-bce6-412b-bcdc-8437c0008466 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:29.080658 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 489/984] 10.222.0.62 () {56 vars in 1374 bytes} [Wed Aug 24 05:58:28 2022] GET /identity/v3/role_assignments?user.id=b82726a12436435cb931b986ef884c2e&scope.project.id=181f4762037e454489d9552cc603179c&role.id=2499f7e98f3e498ab329637f00bb2da4 => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:58:29.149007 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e9ca25-bc24-420f-b3a2-bd0b6ea42092 None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:29.185590 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db42c83f-48c5-4f23-aefc-e59bf76678d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:29.186120 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db42c83f-48c5-4f23-aefc-e59bf76678d0 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:29.186396 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db42c83f-48c5-4f23-aefc-e59bf76678d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:29.186663 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db42c83f-48c5-4f23-aefc-e59bf76678d0 None admin] PATH_INFO: `/v3/services` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:29.187295 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db42c83f-48c5-4f23-aefc-e59bf76678d0 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:58:29.187590 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db42c83f-48c5-4f23-aefc-e59bf76678d0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:29.189020 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db42c83f-48c5-4f23-aefc-e59bf76678d0 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:29.194496 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 496/985] 10.222.0.62 () {56 vars in 1122 bytes} [Wed Aug 24 05:58:29 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:58:29.200060 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0eabaf3-bce6-412b-bcdc-8437c0008466 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:29.212232 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:29.307271 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0eabaf3-bce6-412b-bcdc-8437c0008466 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:29.315926 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db42c83f-48c5-4f23-aefc-e59bf76678d0 None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:29.318826 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:29.343585 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76c5217-9934-4a00-9653-cea86c8e0f84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:29.344194 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76c5217-9934-4a00-9653-cea86c8e0f84 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:29.344468 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76c5217-9934-4a00-9653-cea86c8e0f84 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:29.344803 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76c5217-9934-4a00-9653-cea86c8e0f84 None admin] PATH_INFO: `/v3/services` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:29.345426 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76c5217-9934-4a00-9653-cea86c8e0f84 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:58:29.345724 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76c5217-9934-4a00-9653-cea86c8e0f84 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:29.347074 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76c5217-9934-4a00-9653-cea86c8e0f84 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:29.352168 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 490/986] 10.222.0.62 () {56 vars in 1122 bytes} [Wed Aug 24 05:58:29 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:58:29.358398 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76c5217-9934-4a00-9653-cea86c8e0f84 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:29.428637 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db42c83f-48c5-4f23-aefc-e59bf76678d0 None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:29.441175 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:29.465893 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d90678-8c6c-46c3-974d-9daa2e386d63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:29.466590 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d90678-8c6c-46c3-974d-9daa2e386d63 None admin] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:29.466781 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d90678-8c6c-46c3-974d-9daa2e386d63 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:29.467071 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d90678-8c6c-46c3-974d-9daa2e386d63 None admin] PATH_INFO: `/v3/services` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:29.467809 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d90678-8c6c-46c3-974d-9daa2e386d63 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:29.469206 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d90678-8c6c-46c3-974d-9daa2e386d63 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:29.472329 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76c5217-9934-4a00-9653-cea86c8e0f84 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:29.507242 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a492c4e8-fdc9-4e80-8a1b-d1375bef965c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:29.507846 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a492c4e8-fdc9-4e80-8a1b-d1375bef965c None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:29.508127 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a492c4e8-fdc9-4e80-8a1b-d1375bef965c None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:29.508400 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a492c4e8-fdc9-4e80-8a1b-d1375bef965c None admin] PATH_INFO: `/v3/endpoints` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:29.509130 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a492c4e8-fdc9-4e80-8a1b-d1375bef965c None admin] RBAC: Adding query filter params (interface=admin, service_id=c2fd3d0526d24c39a9cd5fb18573dd75, region_id=RegionOne) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:58:29.509438 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a492c4e8-fdc9-4e80-8a1b-d1375bef965c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:29.510927 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a492c4e8-fdc9-4e80-8a1b-d1375bef965c None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:29.521105 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 491/987] 10.222.0.62 () {56 vars in 1249 bytes} [Wed Aug 24 05:58:29 2022] GET /identity/v3/endpoints?service_id=c2fd3d0526d24c39a9cd5fb18573dd75&interface=admin®ion_id=RegionOne => generated 193 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:58:29.522170 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a492c4e8-fdc9-4e80-8a1b-d1375bef965c None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:29.536151 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 497/988] 10.222.0.62 () {60 vars in 1141 bytes} [Wed Aug 24 05:58:29 2022] POST /identity/v3/services => generated 242 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 05:58:29.639628 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a492c4e8-fdc9-4e80-8a1b-d1375bef965c None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:29.676973 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:29.683152 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40597a17-de57-4255-960f-91d2348ccc94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:29.683766 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40597a17-de57-4255-960f-91d2348ccc94 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:29.684043 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40597a17-de57-4255-960f-91d2348ccc94 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:29.684312 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40597a17-de57-4255-960f-91d2348ccc94 None admin] PATH_INFO: `/v3/services` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:29.685069 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40597a17-de57-4255-960f-91d2348ccc94 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:29.687073 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40597a17-de57-4255-960f-91d2348ccc94 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:29.693925 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 492/989] 10.222.0.62 () {56 vars in 1087 bytes} [Wed Aug 24 05:58:29 2022] GET /identity/v3/services => generated 1727 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:58:31.016137 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060071b7-cd81-4fa8-a911-f290b5704b4c None None] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:31.016663 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060071b7-cd81-4fa8-a911-f290b5704b4c None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:31.016663 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060071b7-cd81-4fa8-a911-f290b5704b4c None None] PATH_INFO: `/` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:31.018103 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 498/990] 10.222.0.62 () {54 vars in 927 bytes} [Wed Aug 24 05:58:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:58:31.023135 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba366dd2-9d4d-449b-8bcc-b94b56e0ba9b None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:31.023521 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba366dd2-9d4d-449b-8bcc-b94b56e0ba9b None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:31.023694 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba366dd2-9d4d-449b-8bcc-b94b56e0ba9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:31.042031 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-ba366dd2-9d4d-449b-8bcc-b94b56e0ba9b None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:58:31.043539 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-ba366dd2-9d4d-449b-8bcc-b94b56e0ba9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:58:31.113104 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 493/991] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Aug 24 05:58:31 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 05:58:31.378355 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060071b7-cd81-4fa8-a911-f290b5704b4c None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:31.444435 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:31.460371 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95615ca3-1b51-4f4c-9a6f-1c1773a3c27c None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:31.460686 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95615ca3-1b51-4f4c-9a6f-1c1773a3c27c None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:31.460938 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95615ca3-1b51-4f4c-9a6f-1c1773a3c27c None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:31.462415 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 494/992] 10.222.0.62 () {54 vars in 927 bytes} [Wed Aug 24 05:58:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 05:58:31.467429 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f2456e-c4d1-4f8b-b88c-dce833769ed8 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:31.467716 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f2456e-c4d1-4f8b-b88c-dce833769ed8 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:31.467992 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f2456e-c4d1-4f8b-b88c-dce833769ed8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:31.486403 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-d4f2456e-c4d1-4f8b-b88c-dce833769ed8 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:58:31.488458 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060071b7-cd81-4fa8-a911-f290b5704b4c None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:31.488604 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-d4f2456e-c4d1-4f8b-b88c-dce833769ed8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:58:31.524535 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88e4022-7abe-47e3-b7c6-181f643b6bbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:31.525172 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88e4022-7abe-47e3-b7c6-181f643b6bbd None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:31.525454 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88e4022-7abe-47e3-b7c6-181f643b6bbd None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:31.525727 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88e4022-7abe-47e3-b7c6-181f643b6bbd None admin] PATH_INFO: `/v3/services/image` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:31.526306 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88e4022-7abe-47e3-b7c6-181f643b6bbd None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:31.527670 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88e4022-7abe-47e3-b7c6-181f643b6bbd None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:31.536295 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application [None req-b88e4022-7abe-47e3-b7c6-181f643b6bbd None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 05:58:31.536295 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:58:31.536295 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 05:58:31.536295 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 05:58:31.536295 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 05:58:31.536295 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 05:58:31.536295 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 05:58:31.536295 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 05:58:31.536295 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 05:58:31.536295 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Aug 24 05:58:31.536295 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 05:58:31.536295 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Aug 24 05:58:31.536295 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:58:31.536295 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:58:31.536295 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:58:31.536295 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:58:31.536295 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:58:31.537519 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:58:31.537519 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:58:31.537519 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:58:31.537519 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:58:31.537519 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:58:31.537519 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:58:31.537519 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 05:58:31.537519 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 05:58:31.537519 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 05:58:31.537519 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 05:58:31.537519 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:58:31.537519 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:58:31.537519 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:58:31.537519 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:58:31.537519 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:58:31.537519 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:58:31.538638 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:58:31.538638 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application with Lock( Aug 24 05:58:31.538638 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:58:31.538638 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:58:31.538638 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:58:31.538638 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:58:31.538638 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:58:31.538638 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:58:31.538638 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:58:31.538638 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:58:31.538638 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 05:58:31.538638 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 05:58:31.538638 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 05:58:31.538638 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Aug 24 05:58:31.542292 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 499/993] 10.222.0.62 () {56 vars in 1105 bytes} [Wed Aug 24 05:58:31 2022] GET /identity/v3/services/image => generated 86 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:58:31.543034 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88e4022-7abe-47e3-b7c6-181f643b6bbd None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:31.571086 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 495/994] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Aug 24 05:58:31 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 05:58:31.572857 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c10b6d-7f15-4d8c-be26-f956750c89be None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:31.573146 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c10b6d-7f15-4d8c-be26-f956750c89be None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:31.573422 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c10b6d-7f15-4d8c-be26-f956750c89be None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:31.574657 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 496/995] 10.222.0.62 () {54 vars in 927 bytes} [Wed Aug 24 05:58:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:58:31.580233 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ca6159-7345-4cdf-aacb-bb229f439ffa None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:31.580534 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ca6159-7345-4cdf-aacb-bb229f439ffa None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:31.580817 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ca6159-7345-4cdf-aacb-bb229f439ffa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:31.599073 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-e7ca6159-7345-4cdf-aacb-bb229f439ffa None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:58:31.600244 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-e7ca6159-7345-4cdf-aacb-bb229f439ffa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:58:31.644555 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88e4022-7abe-47e3-b7c6-181f643b6bbd None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:31.652976 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:31.676527 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 497/996] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Aug 24 05:58:31 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 05:58:31.676686 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:31.682677 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3db1de0-5658-4a5a-8315-e0af58b9b9e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:31.683285 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3db1de0-5658-4a5a-8315-e0af58b9b9e0 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:31.683566 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3db1de0-5658-4a5a-8315-e0af58b9b9e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:31.683837 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3db1de0-5658-4a5a-8315-e0af58b9b9e0 None admin] PATH_INFO: `/v3/services` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:31.684464 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3db1de0-5658-4a5a-8315-e0af58b9b9e0 None admin] RBAC: Adding query filter params (name=image) {{(pid=81851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:58:31.685305 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3db1de0-5658-4a5a-8315-e0af58b9b9e0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:31.686678 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3db1de0-5658-4a5a-8315-e0af58b9b9e0 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:31.691407 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 500/997] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Aug 24 05:58:31 2022] GET /identity/v3/services?name=image => generated 122 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:58:31.697176 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ca6159-7345-4cdf-aacb-bb229f439ffa None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:31.800575 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ca6159-7345-4cdf-aacb-bb229f439ffa None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:31.839052 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3db1de0-5658-4a5a-8315-e0af58b9b9e0 None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:31.928595 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fac9ac-4cd4-4b10-a21e-babc8e9783a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:31.929380 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fac9ac-4cd4-4b10-a21e-babc8e9783a7 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:31.929665 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fac9ac-4cd4-4b10-a21e-babc8e9783a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:31.929942 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fac9ac-4cd4-4b10-a21e-babc8e9783a7 None admin] PATH_INFO: `/v3/services` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:31.930579 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6fac9ac-4cd4-4b10-a21e-babc8e9783a7 None admin] RBAC: Adding query filter params (type=image) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:58:31.930885 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6fac9ac-4cd4-4b10-a21e-babc8e9783a7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:31.932258 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6fac9ac-4cd4-4b10-a21e-babc8e9783a7 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:31.937734 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 498/998] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Aug 24 05:58:31 2022] GET /identity/v3/services?type=image => generated 122 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:58:31.938522 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fac9ac-4cd4-4b10-a21e-babc8e9783a7 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:31.964218 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3db1de0-5658-4a5a-8315-e0af58b9b9e0 None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:31.998254 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:32.004072 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a128ea41-d39e-4e55-9137-77a72cf1b170 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:32.005104 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a128ea41-d39e-4e55-9137-77a72cf1b170 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:32.005599 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a128ea41-d39e-4e55-9137-77a72cf1b170 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:32.005876 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a128ea41-d39e-4e55-9137-77a72cf1b170 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:32.006467 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a128ea41-d39e-4e55-9137-77a72cf1b170 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:32.007867 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a128ea41-d39e-4e55-9137-77a72cf1b170 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:32.017193 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application [None req-a128ea41-d39e-4e55-9137-77a72cf1b170 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 05:58:32.017193 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:58:32.017193 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 05:58:32.017193 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 05:58:32.017193 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 05:58:32.017193 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 05:58:32.017193 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 05:58:32.017193 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 05:58:32.017193 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 05:58:32.017193 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Aug 24 05:58:32.017193 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 05:58:32.017193 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Aug 24 05:58:32.017193 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:58:32.017193 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:58:32.017193 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:58:32.017193 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:58:32.017193 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:58:32.018854 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:58:32.018854 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:58:32.018854 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:58:32.018854 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:58:32.018854 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:58:32.018854 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:58:32.018854 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 05:58:32.018854 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 05:58:32.018854 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 05:58:32.018854 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 05:58:32.018854 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:58:32.018854 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:58:32.018854 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:58:32.018854 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:58:32.018854 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:58:32.018854 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:58:32.019712 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:58:32.019712 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application with Lock( Aug 24 05:58:32.019712 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:58:32.019712 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:58:32.019712 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:58:32.019712 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:58:32.019712 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:58:32.019712 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:58:32.019712 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:58:32.019712 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:58:32.019712 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 05:58:32.019712 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 05:58:32.019712 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 05:58:32.019712 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Aug 24 05:58:32.019712 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 501/999] 10.222.0.62 () {56 vars in 1111 bytes} [Wed Aug 24 05:58:31 2022] GET /identity/v3/services/compute => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 05:58:32.024649 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a128ea41-d39e-4e55-9137-77a72cf1b170 None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:32.058984 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fac9ac-4cd4-4b10-a21e-babc8e9783a7 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:32.089388 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:32.098013 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2f4c94-da6e-431e-9973-c562686cbcae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:32.098642 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2f4c94-da6e-431e-9973-c562686cbcae None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:32.098916 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2f4c94-da6e-431e-9973-c562686cbcae None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:32.103872 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2f4c94-da6e-431e-9973-c562686cbcae None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:32.103872 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2f4c94-da6e-431e-9973-c562686cbcae None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:32.103872 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2f4c94-da6e-431e-9973-c562686cbcae None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:32.111805 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-3e2f4c94-da6e-431e-9973-c562686cbcae None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 05:58:32.111805 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:58:32.111805 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 05:58:32.111805 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 05:58:32.111805 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 05:58:32.111805 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 05:58:32.111805 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 05:58:32.111805 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 05:58:32.111805 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 05:58:32.111805 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 24 05:58:32.111805 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 05:58:32.111805 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 24 05:58:32.111805 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:58:32.111805 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:58:32.111805 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:58:32.111805 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:58:32.111805 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:58:32.112896 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:58:32.112896 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:58:32.112896 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:58:32.112896 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:58:32.112896 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:58:32.112896 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:58:32.112896 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 05:58:32.112896 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 05:58:32.112896 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 05:58:32.112896 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 05:58:32.112896 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:58:32.112896 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:58:32.112896 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:58:32.112896 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:58:32.112896 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:58:32.112896 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:58:32.113916 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:58:32.113916 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 24 05:58:32.113916 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:58:32.113916 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:58:32.113916 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:58:32.113916 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:58:32.113916 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:58:32.113916 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:58:32.113916 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:58:32.113916 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:58:32.113916 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 05:58:32.113916 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 05:58:32.113916 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 05:58:32.113916 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 24 05:58:32.113916 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 499/1000] 10.222.0.62 () {56 vars in 1126 bytes} [Wed Aug 24 05:58:31 2022] GET /identity/v3/services/object-store => generated 93 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:58:32.119087 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2f4c94-da6e-431e-9973-c562686cbcae None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:32.143984 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a128ea41-d39e-4e55-9137-77a72cf1b170 None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:32.182296 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c362aa4c-a19f-4ae5-a455-f1118cfdb06c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:32.182912 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c362aa4c-a19f-4ae5-a455-f1118cfdb06c None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:32.183194 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c362aa4c-a19f-4ae5-a455-f1118cfdb06c None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:32.183465 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c362aa4c-a19f-4ae5-a455-f1118cfdb06c None admin] PATH_INFO: `/v3/services` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:32.184082 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c362aa4c-a19f-4ae5-a455-f1118cfdb06c None admin] RBAC: Adding query filter params (name=compute) {{(pid=81851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:58:32.184381 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c362aa4c-a19f-4ae5-a455-f1118cfdb06c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:32.185875 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c362aa4c-a19f-4ae5-a455-f1118cfdb06c None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:32.190704 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 502/1001] 10.222.0.62 () {56 vars in 1112 bytes} [Wed Aug 24 05:58:32 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:58:32.196159 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c362aa4c-a19f-4ae5-a455-f1118cfdb06c None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:32.225408 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2f4c94-da6e-431e-9973-c562686cbcae None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:32.239807 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:32.260238 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17c27a1-0de0-4409-9102-db7e2aa4263c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:32.264097 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17c27a1-0de0-4409-9102-db7e2aa4263c None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:32.264097 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17c27a1-0de0-4409-9102-db7e2aa4263c None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:32.264097 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17c27a1-0de0-4409-9102-db7e2aa4263c None admin] PATH_INFO: `/v3/services` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:32.264097 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17c27a1-0de0-4409-9102-db7e2aa4263c None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:58:32.264097 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17c27a1-0de0-4409-9102-db7e2aa4263c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:32.264097 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17c27a1-0de0-4409-9102-db7e2aa4263c None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:32.269030 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 500/1002] 10.222.0.62 () {56 vars in 1122 bytes} [Wed Aug 24 05:58:32 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:58:32.274441 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17c27a1-0de0-4409-9102-db7e2aa4263c None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:32.304179 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c362aa4c-a19f-4ae5-a455-f1118cfdb06c None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:32.322665 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:32.340240 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2c836b-05ba-465d-8d20-93a67aa37942 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:32.340878 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2c836b-05ba-465d-8d20-93a67aa37942 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:32.341675 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2c836b-05ba-465d-8d20-93a67aa37942 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:32.341949 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2c836b-05ba-465d-8d20-93a67aa37942 None admin] PATH_INFO: `/v3/services` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:32.342568 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2c836b-05ba-465d-8d20-93a67aa37942 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:58:32.342871 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2c836b-05ba-465d-8d20-93a67aa37942 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:32.344247 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2c836b-05ba-465d-8d20-93a67aa37942 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:32.349215 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 503/1003] 10.222.0.62 () {56 vars in 1112 bytes} [Wed Aug 24 05:58:32 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:58:32.354813 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2c836b-05ba-465d-8d20-93a67aa37942 None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:32.385242 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17c27a1-0de0-4409-9102-db7e2aa4263c None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:32.420735 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c7b7b7-f864-44ce-8b2b-109eab2b6327 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:32.421341 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c7b7b7-f864-44ce-8b2b-109eab2b6327 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:32.421621 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c7b7b7-f864-44ce-8b2b-109eab2b6327 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:32.421890 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c7b7b7-f864-44ce-8b2b-109eab2b6327 None admin] PATH_INFO: `/v3/services` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:32.422507 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c7b7b7-f864-44ce-8b2b-109eab2b6327 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:58:32.422808 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c7b7b7-f864-44ce-8b2b-109eab2b6327 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:32.424268 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c7b7b7-f864-44ce-8b2b-109eab2b6327 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:32.429326 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 501/1004] 10.222.0.62 () {56 vars in 1122 bytes} [Wed Aug 24 05:58:32 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:58:32.435141 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c7b7b7-f864-44ce-8b2b-109eab2b6327 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:32.477576 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2c836b-05ba-465d-8d20-93a67aa37942 None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:32.479887 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:32.514270 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ef1d4c-886f-4761-bfd8-726a9963556f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:32.514876 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ef1d4c-886f-4761-bfd8-726a9963556f None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:32.515148 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ef1d4c-886f-4761-bfd8-726a9963556f None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:32.515415 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ef1d4c-886f-4761-bfd8-726a9963556f None admin] PATH_INFO: `/v3/endpoints` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:32.516102 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ef1d4c-886f-4761-bfd8-726a9963556f None admin] RBAC: Adding query filter params (interface=public, service_id=709c5c092d344ce8a95229ab4f57160d, region_id=RegionOne) {{(pid=81851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:58:32.516403 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ef1d4c-886f-4761-bfd8-726a9963556f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:32.517784 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ef1d4c-886f-4761-bfd8-726a9963556f None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:32.523075 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 504/1005] 10.222.0.62 () {56 vars in 1251 bytes} [Wed Aug 24 05:58:32 2022] GET /identity/v3/endpoints?service_id=709c5c092d344ce8a95229ab4f57160d&interface=public®ion_id=RegionOne => generated 194 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:58:32.528165 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ef1d4c-886f-4761-bfd8-726a9963556f None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:32.543608 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c7b7b7-f864-44ce-8b2b-109eab2b6327 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:32.581088 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:32.581991 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b4da49-c4f9-472e-80c3-de1513315a55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:32.582747 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b4da49-c4f9-472e-80c3-de1513315a55 None admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:32.583030 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b4da49-c4f9-472e-80c3-de1513315a55 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:32.583300 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b4da49-c4f9-472e-80c3-de1513315a55 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:32.583950 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b4da49-c4f9-472e-80c3-de1513315a55 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:32.585362 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b4da49-c4f9-472e-80c3-de1513315a55 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:32.635523 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ef1d4c-886f-4761-bfd8-726a9963556f None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:32.668797 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2280aa32-f7f6-4f9b-b248-9915e68de2b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:32.669420 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2280aa32-f7f6-4f9b-b248-9915e68de2b0 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:32.669694 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2280aa32-f7f6-4f9b-b248-9915e68de2b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:32.670228 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2280aa32-f7f6-4f9b-b248-9915e68de2b0 None admin] PATH_INFO: `/v3/services` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:32.670812 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2280aa32-f7f6-4f9b-b248-9915e68de2b0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:32.672181 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2280aa32-f7f6-4f9b-b248-9915e68de2b0 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:32.678183 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 505/1006] 10.222.0.62 () {56 vars in 1087 bytes} [Wed Aug 24 05:58:32 2022] GET /identity/v3/services => generated 1727 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 05:58:32.700993 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 502/1007] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Aug 24 05:58:32 2022] POST /identity/v3/endpoints => generated 325 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 05:58:33.866147 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2a8a5f-30b7-437a-8d55-6865994f89b9 None None] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:33.866699 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2a8a5f-30b7-437a-8d55-6865994f89b9 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:33.866699 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2a8a5f-30b7-437a-8d55-6865994f89b9 None None] PATH_INFO: `/` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:33.868199 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 506/1008] 10.222.0.62 () {54 vars in 927 bytes} [Wed Aug 24 05:58:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 05:58:33.873234 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb91da8-103e-44f7-a5b9-392a80efe576 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:33.873620 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb91da8-103e-44f7-a5b9-392a80efe576 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:33.873814 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb91da8-103e-44f7-a5b9-392a80efe576 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:33.891567 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-6bb91da8-103e-44f7-a5b9-392a80efe576 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:58:33.892757 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-6bb91da8-103e-44f7-a5b9-392a80efe576 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:58:33.967030 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 503/1009] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Aug 24 05:58:33 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 05:58:34.236769 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2a8a5f-30b7-437a-8d55-6865994f89b9 None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:34.343632 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2a8a5f-30b7-437a-8d55-6865994f89b9 None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:34.355617 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:34.379505 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80aef79-88dc-44d7-9f2d-0b16d3ffc727 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:34.380114 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80aef79-88dc-44d7-9f2d-0b16d3ffc727 None admin] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:34.380440 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80aef79-88dc-44d7-9f2d-0b16d3ffc727 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:34.380759 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80aef79-88dc-44d7-9f2d-0b16d3ffc727 None admin] PATH_INFO: `/v3/services` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:34.381389 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80aef79-88dc-44d7-9f2d-0b16d3ffc727 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:34.382820 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80aef79-88dc-44d7-9f2d-0b16d3ffc727 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:34.492616 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 507/1010] 10.222.0.62 () {60 vars in 1141 bytes} [Wed Aug 24 05:58:34 2022] POST /identity/v3/services => generated 242 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:58:34.586976 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ede7b4-bdbe-4ce1-b520-b75e439a399b None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:34.587304 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ede7b4-bdbe-4ce1-b520-b75e439a399b None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:34.587519 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ede7b4-bdbe-4ce1-b520-b75e439a399b None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:34.589089 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 504/1011] 10.222.0.62 () {54 vars in 927 bytes} [Wed Aug 24 05:58:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:58:34.594629 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59aa8b55-5629-4869-8996-b2d684f50003 None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:34.594920 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59aa8b55-5629-4869-8996-b2d684f50003 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:34.595177 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59aa8b55-5629-4869-8996-b2d684f50003 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:34.613878 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-59aa8b55-5629-4869-8996-b2d684f50003 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:58:34.615081 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-59aa8b55-5629-4869-8996-b2d684f50003 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:58:34.636315 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3af41d-d710-433a-9abf-b91bb7f3e39c None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:34.636672 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3af41d-d710-433a-9abf-b91bb7f3e39c None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:34.636882 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3af41d-d710-433a-9abf-b91bb7f3e39c None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:34.639262 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 505/1012] 10.222.0.62 () {54 vars in 927 bytes} [Wed Aug 24 05:58:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 05:58:34.644208 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646c2463-7b3e-4574-9f26-d39dea16841c None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:34.644484 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646c2463-7b3e-4574-9f26-d39dea16841c None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:34.644798 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646c2463-7b3e-4574-9f26-d39dea16841c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:34.664758 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-646c2463-7b3e-4574-9f26-d39dea16841c None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:58:34.665941 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-646c2463-7b3e-4574-9f26-d39dea16841c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:58:34.691003 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:34.700364 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 508/1013] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Aug 24 05:58:34 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 05:58:34.738762 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 506/1014] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Aug 24 05:58:34 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 05:58:34.965916 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59aa8b55-5629-4869-8996-b2d684f50003 None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:35.012690 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646c2463-7b3e-4574-9f26-d39dea16841c None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:35.065581 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:35.071171 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59aa8b55-5629-4869-8996-b2d684f50003 None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:35.105283 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4eda5c7-693c-4963-b729-cebc8777cef0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:35.106361 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4eda5c7-693c-4963-b729-cebc8777cef0 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:35.107172 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4eda5c7-693c-4963-b729-cebc8777cef0 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:35.107598 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4eda5c7-693c-4963-b729-cebc8777cef0 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:35.108179 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4eda5c7-693c-4963-b729-cebc8777cef0 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:35.109588 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4eda5c7-693c-4963-b729-cebc8777cef0 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:35.118241 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646c2463-7b3e-4574-9f26-d39dea16841c None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:35.118407 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application [None req-e4eda5c7-693c-4963-b729-cebc8777cef0 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 05:58:35.118407 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:58:35.118407 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 05:58:35.118407 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 05:58:35.118407 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 05:58:35.118407 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 05:58:35.118407 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 05:58:35.118407 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 05:58:35.118407 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 05:58:35.118407 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Aug 24 05:58:35.118407 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 05:58:35.118407 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Aug 24 05:58:35.118407 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:58:35.118407 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:58:35.118407 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:58:35.118407 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:58:35.118407 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:58:35.119520 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:58:35.119520 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:58:35.119520 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:58:35.119520 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:58:35.119520 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:58:35.119520 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:58:35.119520 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 05:58:35.119520 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 05:58:35.119520 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 05:58:35.119520 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 05:58:35.119520 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:58:35.119520 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:58:35.119520 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:58:35.119520 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:58:35.119520 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:58:35.119520 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:58:35.120554 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:58:35.120554 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application with Lock( Aug 24 05:58:35.120554 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:58:35.120554 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:58:35.120554 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:58:35.120554 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:58:35.120554 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:58:35.120554 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:58:35.120554 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:58:35.120554 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:58:35.120554 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 05:58:35.120554 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 05:58:35.120554 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 05:58:35.120554 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Aug 24 05:58:35.120554 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 509/1015] 10.222.0.62 () {56 vars in 1111 bytes} [Wed Aug 24 05:58:34 2022] GET /identity/v3/services/compute => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:58:35.125696 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4eda5c7-693c-4963-b729-cebc8777cef0 None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:35.159051 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a541985-03e2-4434-963b-28a248c41e6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:35.159681 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a541985-03e2-4434-963b-28a248c41e6a None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:35.159961 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a541985-03e2-4434-963b-28a248c41e6a None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:35.160457 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a541985-03e2-4434-963b-28a248c41e6a None admin] PATH_INFO: `/v3/domains/default` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:35.161455 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a541985-03e2-4434-963b-28a248c41e6a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:35.162892 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a541985-03e2-4434-963b-28a248c41e6a None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:35.164638 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 507/1016] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Aug 24 05:58:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:58:35.170973 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a541985-03e2-4434-963b-28a248c41e6a None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:35.199466 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:35.232855 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4eda5c7-693c-4963-b729-cebc8777cef0 None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:35.267074 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45eaf1ce-220b-45f2-bfa7-710ef8d1d5cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:35.267703 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45eaf1ce-220b-45f2-bfa7-710ef8d1d5cc None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:35.267980 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45eaf1ce-220b-45f2-bfa7-710ef8d1d5cc None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:35.268254 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45eaf1ce-220b-45f2-bfa7-710ef8d1d5cc None admin] PATH_INFO: `/v3/services` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:35.268905 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45eaf1ce-220b-45f2-bfa7-710ef8d1d5cc None admin] RBAC: Adding query filter params (name=compute) {{(pid=81851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:58:35.269210 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45eaf1ce-220b-45f2-bfa7-710ef8d1d5cc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:35.270562 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45eaf1ce-220b-45f2-bfa7-710ef8d1d5cc None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:35.280331 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 510/1017] 10.222.0.62 () {56 vars in 1112 bytes} [Wed Aug 24 05:58:35 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:58:35.280471 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a541985-03e2-4434-963b-28a248c41e6a None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:35.281425 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45eaf1ce-220b-45f2-bfa7-710ef8d1d5cc None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:35.309435 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:35.315095 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e4e5b6-d181-4054-b175-b5b2eb703180 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:35.315685 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e4e5b6-d181-4054-b175-b5b2eb703180 None admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:35.315961 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e4e5b6-d181-4054-b175-b5b2eb703180 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:35.316227 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e4e5b6-d181-4054-b175-b5b2eb703180 None admin] PATH_INFO: `/v3/projects` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:35.316885 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e4e5b6-d181-4054-b175-b5b2eb703180 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:35.318291 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e4e5b6-d181-4054-b175-b5b2eb703180 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:35.388353 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45eaf1ce-220b-45f2-bfa7-710ef8d1d5cc None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:35.424383 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9d21fa-4db6-451f-b78c-1bf9762d9706 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:35.424722 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 508/1018] 10.222.0.62 () {60 vars in 1141 bytes} [Wed Aug 24 05:58:35 2022] POST /identity/v3/projects => generated 311 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 05:58:35.425421 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9d21fa-4db6-451f-b78c-1bf9762d9706 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:35.425703 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9d21fa-4db6-451f-b78c-1bf9762d9706 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:35.425974 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9d21fa-4db6-451f-b78c-1bf9762d9706 None admin] PATH_INFO: `/v3/services` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:35.426624 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9d21fa-4db6-451f-b78c-1bf9762d9706 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:58:35.426927 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9d21fa-4db6-451f-b78c-1bf9762d9706 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:35.428342 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9d21fa-4db6-451f-b78c-1bf9762d9706 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:35.434328 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 511/1019] 10.222.0.62 () {56 vars in 1112 bytes} [Wed Aug 24 05:58:35 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:58:35.449876 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e4e5b6-d181-4054-b175-b5b2eb703180 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:35.538450 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:35.562268 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e4e5b6-d181-4054-b175-b5b2eb703180 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:35.602101 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc601438-bd9b-4602-ab1d-fd1e3b7e03ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:35.602902 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc601438-bd9b-4602-ab1d-fd1e3b7e03ae None admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:35.603401 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc601438-bd9b-4602-ab1d-fd1e3b7e03ae None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:35.603854 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc601438-bd9b-4602-ab1d-fd1e3b7e03ae None admin] PATH_INFO: `/v3/endpoints` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:35.604993 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc601438-bd9b-4602-ab1d-fd1e3b7e03ae None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:35.606647 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc601438-bd9b-4602-ab1d-fd1e3b7e03ae None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:35.725040 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 509/1020] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Aug 24 05:58:35 2022] POST /identity/v3/endpoints => generated 334 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:58:36.418731 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8913b4dc-f63f-4a1c-8e5b-f81dd80d60ab None None] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:36.419810 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8913b4dc-f63f-4a1c-8e5b-f81dd80d60ab None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:36.419810 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8913b4dc-f63f-4a1c-8e5b-f81dd80d60ab None None] PATH_INFO: `/` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:36.421446 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 512/1021] 10.222.0.62 () {54 vars in 927 bytes} [Wed Aug 24 05:58:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:58:36.426677 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72eacf7b-8fd3-4913-9bd4-19e8bf817d10 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:36.427098 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72eacf7b-8fd3-4913-9bd4-19e8bf817d10 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:36.427270 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72eacf7b-8fd3-4913-9bd4-19e8bf817d10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:36.446202 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-72eacf7b-8fd3-4913-9bd4-19e8bf817d10 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:58:36.447379 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-72eacf7b-8fd3-4913-9bd4-19e8bf817d10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:58:36.524876 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 510/1022] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Aug 24 05:58:36 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 05:58:36.789627 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8913b4dc-f63f-4a1c-8e5b-f81dd80d60ab None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:36.883525 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:36.897204 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8913b4dc-f63f-4a1c-8e5b-f81dd80d60ab None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:36.931884 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c607c1c-1d91-4cef-8532-11f510e24213 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:36.932514 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c607c1c-1d91-4cef-8532-11f510e24213 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:36.932817 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c607c1c-1d91-4cef-8532-11f510e24213 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:36.933083 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c607c1c-1d91-4cef-8532-11f510e24213 None admin] PATH_INFO: `/v3/services/image` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:36.933652 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c607c1c-1d91-4cef-8532-11f510e24213 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:36.935535 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c607c1c-1d91-4cef-8532-11f510e24213 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:36.943977 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application [None req-1c607c1c-1d91-4cef-8532-11f510e24213 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 05:58:36.943977 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:58:36.943977 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 05:58:36.943977 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 05:58:36.943977 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 05:58:36.943977 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 05:58:36.943977 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 05:58:36.943977 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 05:58:36.943977 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 05:58:36.943977 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Aug 24 05:58:36.943977 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 05:58:36.943977 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Aug 24 05:58:36.943977 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:58:36.943977 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:58:36.943977 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:58:36.943977 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:58:36.943977 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:58:36.944981 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:58:36.944981 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:58:36.944981 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:58:36.944981 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:58:36.944981 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:58:36.944981 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:58:36.944981 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 05:58:36.944981 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 05:58:36.944981 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 05:58:36.944981 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 05:58:36.944981 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:58:36.944981 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:58:36.944981 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:58:36.944981 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:58:36.944981 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:58:36.944981 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:58:36.949526 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:58:36.949526 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application with Lock( Aug 24 05:58:36.949526 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:58:36.949526 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:58:36.949526 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:58:36.949526 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:58:36.949526 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:58:36.949526 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:58:36.949526 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:58:36.949526 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:58:36.949526 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 05:58:36.949526 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 05:58:36.949526 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 05:58:36.949526 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Aug 24 05:58:36.949526 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 513/1023] 10.222.0.62 () {56 vars in 1105 bytes} [Wed Aug 24 05:58:36 2022] GET /identity/v3/services/image => generated 86 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 05:58:36.952022 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72eacf7b-8fd3-4913-9bd4-19e8bf817d10 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:37.011800 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:37.052925 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72eacf7b-8fd3-4913-9bd4-19e8bf817d10 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:37.086875 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3edba0d-b402-4b9c-8a82-7a33b3cd5ab8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:37.087486 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3edba0d-b402-4b9c-8a82-7a33b3cd5ab8 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:37.087759 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3edba0d-b402-4b9c-8a82-7a33b3cd5ab8 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:37.088027 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3edba0d-b402-4b9c-8a82-7a33b3cd5ab8 None admin] PATH_INFO: `/v3/services` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:37.088674 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3edba0d-b402-4b9c-8a82-7a33b3cd5ab8 None admin] RBAC: Adding query filter params (name=image) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:58:37.088975 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3edba0d-b402-4b9c-8a82-7a33b3cd5ab8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:37.090408 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3edba0d-b402-4b9c-8a82-7a33b3cd5ab8 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:37.099804 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 511/1024] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Aug 24 05:58:36 2022] GET /identity/v3/services?name=image => generated 122 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:58:37.101366 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c607c1c-1d91-4cef-8532-11f510e24213 None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:37.202230 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c607c1c-1d91-4cef-8532-11f510e24213 None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:37.234815 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b40ff3-ea2e-4c20-8dc1-b3be72d85cde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:37.235416 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b40ff3-ea2e-4c20-8dc1-b3be72d85cde None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:37.235688 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b40ff3-ea2e-4c20-8dc1-b3be72d85cde None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:37.235954 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b40ff3-ea2e-4c20-8dc1-b3be72d85cde None admin] PATH_INFO: `/v3/services` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:37.236718 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b40ff3-ea2e-4c20-8dc1-b3be72d85cde None admin] RBAC: Adding query filter params (type=image) {{(pid=81851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:58:37.237026 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b40ff3-ea2e-4c20-8dc1-b3be72d85cde None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:37.238373 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b40ff3-ea2e-4c20-8dc1-b3be72d85cde None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:37.243286 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 514/1025] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Aug 24 05:58:37 2022] GET /identity/v3/services?type=image => generated 350 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:58:37.249752 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3edba0d-b402-4b9c-8a82-7a33b3cd5ab8 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:37.349752 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3edba0d-b402-4b9c-8a82-7a33b3cd5ab8 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:37.372853 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:37.383815 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26c7f12-887e-477d-9481-b5635464a934 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:37.388342 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26c7f12-887e-477d-9481-b5635464a934 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:37.388342 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26c7f12-887e-477d-9481-b5635464a934 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:37.388342 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26c7f12-887e-477d-9481-b5635464a934 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:37.388342 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26c7f12-887e-477d-9481-b5635464a934 None admin] RBAC: Adding query filter params (interface=public, service_id=93564159db074202b991861e61fcc97d, region_id=RegionOne) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:58:37.388342 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26c7f12-887e-477d-9481-b5635464a934 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:37.388342 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26c7f12-887e-477d-9481-b5635464a934 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:37.393023 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 512/1026] 10.222.0.62 () {56 vars in 1251 bytes} [Wed Aug 24 05:58:37 2022] GET /identity/v3/endpoints?service_id=93564159db074202b991861e61fcc97d&interface=public®ion_id=RegionOne => generated 194 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:58:37.398675 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b40ff3-ea2e-4c20-8dc1-b3be72d85cde None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:37.406674 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bce34c-4da3-4873-8353-07a0cb00c79e None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:37.406961 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bce34c-4da3-4873-8353-07a0cb00c79e None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:37.407234 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bce34c-4da3-4873-8353-07a0cb00c79e None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:37.409596 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 513/1027] 10.222.0.62 () {54 vars in 927 bytes} [Wed Aug 24 05:58:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 05:58:37.413702 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01345a74-e743-4b35-954c-d3651e59c350 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:37.413982 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01345a74-e743-4b35-954c-d3651e59c350 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:37.414265 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01345a74-e743-4b35-954c-d3651e59c350 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:37.419456 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:37.432355 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-01345a74-e743-4b35-954c-d3651e59c350 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:58:37.433551 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-01345a74-e743-4b35-954c-d3651e59c350 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:58:37.505507 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b40ff3-ea2e-4c20-8dc1-b3be72d85cde None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:37.516531 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 514/1028] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Aug 24 05:58:37 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 05:58:37.539265 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406d3f5d-e2e2-4347-852a-a9b772a49e7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:37.539864 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406d3f5d-e2e2-4347-852a-a9b772a49e7c None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:37.540138 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406d3f5d-e2e2-4347-852a-a9b772a49e7c None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:37.540409 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406d3f5d-e2e2-4347-852a-a9b772a49e7c None admin] PATH_INFO: `/v3/services` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:37.541026 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406d3f5d-e2e2-4347-852a-a9b772a49e7c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:37.542373 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406d3f5d-e2e2-4347-852a-a9b772a49e7c None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:37.549116 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 515/1029] 10.222.0.62 () {56 vars in 1087 bytes} [Wed Aug 24 05:58:37 2022] GET /identity/v3/services => generated 1957 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:58:37.735072 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcf7a35-cbeb-4cd6-9a38-6c0b6209adf9 None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:37.735360 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcf7a35-cbeb-4cd6-9a38-6c0b6209adf9 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:37.735635 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcf7a35-cbeb-4cd6-9a38-6c0b6209adf9 None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:37.737030 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 515/1030] 10.222.0.62 () {54 vars in 927 bytes} [Wed Aug 24 05:58:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:58:37.742147 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1edd9a-0983-43e1-9f24-0d4b188d3696 None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:37.742487 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1edd9a-0983-43e1-9f24-0d4b188d3696 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:37.742768 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1edd9a-0983-43e1-9f24-0d4b188d3696 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:37.761596 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-7b1edd9a-0983-43e1-9f24-0d4b188d3696 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:58:37.763322 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-7b1edd9a-0983-43e1-9f24-0d4b188d3696 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:58:37.778849 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcf7a35-cbeb-4cd6-9a38-6c0b6209adf9 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:37.840135 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 516/1031] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Aug 24 05:58:37 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 05:58:37.894871 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcf7a35-cbeb-4cd6-9a38-6c0b6209adf9 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:37.913513 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:37.930350 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3395f3e-7323-489d-8d8b-70e53498c3a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:37.931013 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3395f3e-7323-489d-8d8b-70e53498c3a9 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:37.931299 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3395f3e-7323-489d-8d8b-70e53498c3a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:37.931576 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3395f3e-7323-489d-8d8b-70e53498c3a9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:37.932702 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3395f3e-7323-489d-8d8b-70e53498c3a9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:37.934153 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3395f3e-7323-489d-8d8b-70e53498c3a9 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:37.936674 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 516/1032] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Aug 24 05:58:37 2022] GET /identity/v3/domains/default => generated 199 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:58:37.942156 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1edd9a-0983-43e1-9f24-0d4b188d3696 None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:37.954132 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:38.042240 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1edd9a-0983-43e1-9f24-0d4b188d3696 None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:38.076946 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7045273f-1f10-4b5a-a7b8-2761e1622904 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:38.077552 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7045273f-1f10-4b5a-a7b8-2761e1622904 None admin] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:38.077834 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7045273f-1f10-4b5a-a7b8-2761e1622904 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:38.078109 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7045273f-1f10-4b5a-a7b8-2761e1622904 None admin] PATH_INFO: `/v3/users` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:38.078756 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7045273f-1f10-4b5a-a7b8-2761e1622904 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:38.080184 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7045273f-1f10-4b5a-a7b8-2761e1622904 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:38.115247 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3395f3e-7323-489d-8d8b-70e53498c3a9 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:38.192455 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 517/1033] 10.222.0.62 () {60 vars in 1132 bytes} [Wed Aug 24 05:58:37 2022] POST /identity/v3/users => generated 285 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 05:58:38.220205 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3395f3e-7323-489d-8d8b-70e53498c3a9 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:38.258054 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191aa848-c4e4-4039-8a78-dca114e06814 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:38.258698 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191aa848-c4e4-4039-8a78-dca114e06814 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:38.258974 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191aa848-c4e4-4039-8a78-dca114e06814 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:38.259333 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191aa848-c4e4-4039-8a78-dca114e06814 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:38.270950 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191aa848-c4e4-4039-8a78-dca114e06814 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:58:38.275666 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-191aa848-c4e4-4039-8a78-dca114e06814 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 05:58:38.275666 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:58:38.275666 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:58:38.275666 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:58:38.275666 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:58:38.275666 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:58:38.275666 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:58:38.275666 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:58:38.275666 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:58:38.275666 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:58:38.275666 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:58:38.275666 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:58:38.275666 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:58:38.275666 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:58:38.275666 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:58:38.275666 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:58:38.276665 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:58:38.276665 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:58:38.276665 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:58:38.276665 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:58:38.276665 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:58:38.276665 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:58:38.276665 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:58:38.276665 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:58:38.276665 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:58:38.276665 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:58:38.276665 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:58:38.276665 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 24 05:58:38.276665 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:58:38.276665 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:58:38.276665 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:58:38.276665 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:58:38.277669 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:58:38.277669 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:58:38.277669 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:58:38.277669 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:58:38.277669 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:58:38.277669 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:58:38.277669 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:58:38.277669 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:58:38.277669 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:58:38.277669 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:58:38.277669 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 05:58:38.277669 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 24 05:58:38.277669 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 517/1034] 10.222.0.62 () {56 vars in 1120 bytes} [Wed Aug 24 05:58:38 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 05:58:38.285642 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7045273f-1f10-4b5a-a7b8-2761e1622904 None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:38.356269 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:38.394671 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7045273f-1f10-4b5a-a7b8-2761e1622904 None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:38.430215 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f52c75-7fb0-4e00-aa4e-af06b2e9d756 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:38.430871 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f52c75-7fb0-4e00-aa4e-af06b2e9d756 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:38.431093 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f52c75-7fb0-4e00-aa4e-af06b2e9d756 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:38.431363 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f52c75-7fb0-4e00-aa4e-af06b2e9d756 None admin] PATH_INFO: `/v3/roles` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:38.432039 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f52c75-7fb0-4e00-aa4e-af06b2e9d756 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:58:38.432346 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f52c75-7fb0-4e00-aa4e-af06b2e9d756 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:38.433749 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f52c75-7fb0-4e00-aa4e-af06b2e9d756 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:38.443432 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 518/1035] 10.222.0.62 () {56 vars in 1115 bytes} [Wed Aug 24 05:58:38 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:58:38.449346 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191aa848-c4e4-4039-8a78-dca114e06814 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:38.499923 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:38.553284 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191aa848-c4e4-4039-8a78-dca114e06814 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:38.586852 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f9073b-fd8e-4bab-81f8-f5ff07278545 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:38.587478 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f9073b-fd8e-4bab-81f8-f5ff07278545 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:38.587750 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f9073b-fd8e-4bab-81f8-f5ff07278545 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:38.588013 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f9073b-fd8e-4bab-81f8-f5ff07278545 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:38.598416 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f9073b-fd8e-4bab-81f8-f5ff07278545 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:38.599851 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f9073b-fd8e-4bab-81f8-f5ff07278545 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:38.614415 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-a7f9073b-fd8e-4bab-81f8-f5ff07278545 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:58:38.614415 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:58:38.614415 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:58:38.614415 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:58:38.614415 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:58:38.614415 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:58:38.614415 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:58:38.614415 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:58:38.614415 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:58:38.614415 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:58:38.614415 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:58:38.614415 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:58:38.614415 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 05:58:38.614415 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 05:58:38.614415 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 05:58:38.614415 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 05:58:38.615411 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:58:38.615411 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:58:38.615411 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:58:38.615411 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:58:38.615411 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:58:38.615411 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:58:38.615411 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:58:38.615411 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 24 05:58:38.615411 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:58:38.615411 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:58:38.615411 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:58:38.615411 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:58:38.615411 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:58:38.615411 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:58:38.615411 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:58:38.615411 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:58:38.615411 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 05:58:38.616320 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 05:58:38.616320 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:58:38.616320 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 24 05:58:38.620539 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 518/1036] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Aug 24 05:58:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:58:38.621704 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f52c75-7fb0-4e00-aa4e-af06b2e9d756 None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:38.731237 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f52c75-7fb0-4e00-aa4e-af06b2e9d756 None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:38.757625 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:38.765943 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248f0746-72be-4748-a72e-f43f46dc791d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:38.766548 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248f0746-72be-4748-a72e-f43f46dc791d None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:38.772034 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248f0746-72be-4748-a72e-f43f46dc791d None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:38.772034 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248f0746-72be-4748-a72e-f43f46dc791d None admin] PATH_INFO: `/v3/domains` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:38.772034 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248f0746-72be-4748-a72e-f43f46dc791d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:58:38.772034 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248f0746-72be-4748-a72e-f43f46dc791d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:38.772034 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248f0746-72be-4748-a72e-f43f46dc791d None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:38.786235 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 519/1037] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Aug 24 05:58:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:58:38.787924 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f9073b-fd8e-4bab-81f8-f5ff07278545 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:38.888435 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:38.892614 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f9073b-fd8e-4bab-81f8-f5ff07278545 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:38.927218 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf18771-2c81-499f-bca4-3aa0117d049b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:38.927850 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf18771-2c81-499f-bca4-3aa0117d049b None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:38.928132 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf18771-2c81-499f-bca4-3aa0117d049b None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:38.928408 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf18771-2c81-499f-bca4-3aa0117d049b None admin] PATH_INFO: `/v3/users/nova` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:38.935905 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf18771-2c81-499f-bca4-3aa0117d049b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:38.937370 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf18771-2c81-499f-bca4-3aa0117d049b None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:38.948611 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-5cf18771-2c81-499f-bca4-3aa0117d049b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 05:58:38.948611 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:58:38.948611 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:58:38.948611 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:58:38.948611 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:58:38.948611 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:58:38.948611 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:58:38.948611 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:58:38.948611 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:58:38.948611 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:58:38.948611 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:58:38.948611 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:58:38.948611 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 05:58:38.948611 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 05:58:38.948611 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 05:58:38.948611 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 05:58:38.949637 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:58:38.949637 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:58:38.949637 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 05:58:38.949637 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 05:58:38.949637 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 05:58:38.949637 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 05:58:38.949637 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:58:38.949637 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:58:38.949637 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:58:38.949637 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:58:38.949637 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:58:38.949637 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 24 05:58:38.949637 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:58:38.949637 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:58:38.949637 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:58:38.949637 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:58:38.949637 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:58:38.949637 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:58:38.951271 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:58:38.951271 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:58:38.951271 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 05:58:38.951271 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 05:58:38.951271 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 05:58:38.951271 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 05:58:38.951271 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 05:58:38.951271 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 05:58:38.951271 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 05:58:38.951271 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 24 05:58:38.952590 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 519/1038] 10.222.0.62 () {56 vars in 1093 bytes} [Wed Aug 24 05:58:38 2022] GET /identity/v3/users/nova => generated 82 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:58:38.956294 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248f0746-72be-4748-a72e-f43f46dc791d None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:39.056654 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248f0746-72be-4748-a72e-f43f46dc791d None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:39.091041 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5354abac-567e-4498-b723-522b7d29870c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:39.091645 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5354abac-567e-4498-b723-522b7d29870c None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:39.091927 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5354abac-567e-4498-b723-522b7d29870c None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:39.092208 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5354abac-567e-4498-b723-522b7d29870c None admin] PATH_INFO: `/v3/users` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:39.092971 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5354abac-567e-4498-b723-522b7d29870c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:58:39.093282 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5354abac-567e-4498-b723-522b7d29870c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:39.094712 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5354abac-567e-4498-b723-522b7d29870c None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:39.116666 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 520/1039] 10.222.0.62 () {56 vars in 1133 bytes} [Wed Aug 24 05:58:38 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:58:39.117931 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf18771-2c81-499f-bca4-3aa0117d049b None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:39.221946 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf18771-2c81-499f-bca4-3aa0117d049b None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:39.255713 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d19d5b5-b63b-4599-a2be-df90e87d8945 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:39.256315 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d19d5b5-b63b-4599-a2be-df90e87d8945 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:39.256622 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d19d5b5-b63b-4599-a2be-df90e87d8945 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:39.256894 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d19d5b5-b63b-4599-a2be-df90e87d8945 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:39.283677 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:39.326682 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d19d5b5-b63b-4599-a2be-df90e87d8945 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:39.328381 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d19d5b5-b63b-4599-a2be-df90e87d8945 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:39.433138 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 520/1040] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Aug 24 05:58:39 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 05:58:39.434406 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5354abac-567e-4498-b723-522b7d29870c None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:39.454955 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cd80b6-d40e-415d-8685-4bbf4eb34883 None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:39.455420 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cd80b6-d40e-415d-8685-4bbf4eb34883 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:39.455521 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cd80b6-d40e-415d-8685-4bbf4eb34883 None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:39.456954 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 521/1041] 10.222.0.62 () {54 vars in 927 bytes} [Wed Aug 24 05:58:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 05:58:39.461488 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8d6800-9a1a-4341-9ef4-b429a1a57eb2 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:39.461765 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8d6800-9a1a-4341-9ef4-b429a1a57eb2 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:39.462036 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8d6800-9a1a-4341-9ef4-b429a1a57eb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:39.480214 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-5b8d6800-9a1a-4341-9ef4-b429a1a57eb2 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:58:39.481373 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-5b8d6800-9a1a-4341-9ef4-b429a1a57eb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:58:39.484100 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:39.541857 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5354abac-567e-4498-b723-522b7d29870c None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:39.547188 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:39.561058 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 522/1042] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Aug 24 05:58:39 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 05:58:39.582029 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7121fc70-77a7-4f7d-be3c-7ea1f3a9391d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:39.582655 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7121fc70-77a7-4f7d-be3c-7ea1f3a9391d None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:39.582929 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7121fc70-77a7-4f7d-be3c-7ea1f3a9391d None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:39.583197 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7121fc70-77a7-4f7d-be3c-7ea1f3a9391d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:39.593942 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7121fc70-77a7-4f7d-be3c-7ea1f3a9391d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:39.595396 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7121fc70-77a7-4f7d-be3c-7ea1f3a9391d None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:39.609866 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application [None req-7121fc70-77a7-4f7d-be3c-7ea1f3a9391d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:58:39.609866 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:58:39.609866 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:58:39.609866 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:58:39.609866 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:58:39.609866 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:58:39.609866 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:58:39.609866 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:58:39.609866 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:58:39.609866 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:58:39.609866 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:58:39.609866 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:58:39.609866 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 05:58:39.609866 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 05:58:39.609866 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 05:58:39.609866 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 05:58:39.610815 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:58:39.610815 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:58:39.610815 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:58:39.610815 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:58:39.610815 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:58:39.610815 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:58:39.610815 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:58:39.610815 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application with Lock( Aug 24 05:58:39.610815 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:58:39.610815 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:58:39.610815 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:58:39.610815 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:58:39.610815 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:58:39.610815 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:58:39.610815 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:58:39.610815 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:58:39.610815 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 05:58:39.611726 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 05:58:39.611726 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:58:39.611726 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Aug 24 05:58:39.611726 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 521/1043] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Aug 24 05:58:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:58:39.623037 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8d6800-9a1a-4341-9ef4-b429a1a57eb2 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:39.725070 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8d6800-9a1a-4341-9ef4-b429a1a57eb2 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:39.759815 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782e9bad-865f-4bd4-a16d-71cfd6d55cd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:39.760375 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782e9bad-865f-4bd4-a16d-71cfd6d55cd3 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:39.760697 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782e9bad-865f-4bd4-a16d-71cfd6d55cd3 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:39.760974 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782e9bad-865f-4bd4-a16d-71cfd6d55cd3 None admin] PATH_INFO: `/v3/domains` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:39.761621 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782e9bad-865f-4bd4-a16d-71cfd6d55cd3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:58:39.762013 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782e9bad-865f-4bd4-a16d-71cfd6d55cd3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:39.763465 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782e9bad-865f-4bd4-a16d-71cfd6d55cd3 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:39.780394 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 523/1044] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Aug 24 05:58:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:58:39.781286 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7121fc70-77a7-4f7d-be3c-7ea1f3a9391d None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:39.831099 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782e9bad-865f-4bd4-a16d-71cfd6d55cd3 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:39.880732 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:39.887892 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:39.892252 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7121fc70-77a7-4f7d-be3c-7ea1f3a9391d None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:39.928928 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe64c8d6-9ec3-4132-a071-988be1687865 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:39.930047 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe64c8d6-9ec3-4132-a071-988be1687865 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:39.930538 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe64c8d6-9ec3-4132-a071-988be1687865 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:39.931014 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe64c8d6-9ec3-4132-a071-988be1687865 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:39.932624 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782e9bad-865f-4bd4-a16d-71cfd6d55cd3 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:39.938823 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe64c8d6-9ec3-4132-a071-988be1687865 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:39.940707 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe64c8d6-9ec3-4132-a071-988be1687865 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:39.953922 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application [None req-fe64c8d6-9ec3-4132-a071-988be1687865 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 05:58:39.953922 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:58:39.953922 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:58:39.953922 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:58:39.953922 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:58:39.953922 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:58:39.953922 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:58:39.953922 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:58:39.953922 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:58:39.953922 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:58:39.953922 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:58:39.953922 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:58:39.953922 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 05:58:39.953922 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 05:58:39.953922 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 05:58:39.954908 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 05:58:39.954908 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:58:39.954908 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:58:39.954908 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:58:39.954908 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:58:39.954908 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:58:39.954908 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:58:39.954908 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:58:39.954908 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application with Lock( Aug 24 05:58:39.954908 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:58:39.954908 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:58:39.954908 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:58:39.954908 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:58:39.954908 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:58:39.954908 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:58:39.954908 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:58:39.954908 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:58:39.955836 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 05:58:39.955836 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 05:58:39.955836 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 05:58:39.955836 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 05:58:39.955836 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 05:58:39.955836 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 05:58:39.955836 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 05:58:39.955836 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Aug 24 05:58:39.960053 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 522/1045] 10.222.0.62 () {56 vars in 1111 bytes} [Wed Aug 24 05:58:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 05:58:39.960769 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe64c8d6-9ec3-4132-a071-988be1687865 None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:39.971373 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907b3415-26f6-4fae-a551-3a07be90b615 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:39.972009 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907b3415-26f6-4fae-a551-3a07be90b615 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:39.972347 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907b3415-26f6-4fae-a551-3a07be90b615 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:39.972658 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907b3415-26f6-4fae-a551-3a07be90b615 None admin] PATH_INFO: `/v3/services/image` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:39.973292 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907b3415-26f6-4fae-a551-3a07be90b615 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:39.974669 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907b3415-26f6-4fae-a551-3a07be90b615 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:39.983551 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-907b3415-26f6-4fae-a551-3a07be90b615 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 05:58:39.983551 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:58:39.983551 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 05:58:39.983551 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 05:58:39.983551 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 05:58:39.983551 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 05:58:39.983551 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 05:58:39.983551 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 05:58:39.983551 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 05:58:39.983551 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 24 05:58:39.983551 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 05:58:39.983551 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 24 05:58:39.983551 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:58:39.983551 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:58:39.983551 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:58:39.983551 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:58:39.983551 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:58:39.984844 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:58:39.984844 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:58:39.984844 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:58:39.984844 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:58:39.984844 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:58:39.984844 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:58:39.984844 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 05:58:39.984844 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 05:58:39.984844 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 05:58:39.984844 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 05:58:39.984844 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:58:39.984844 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:58:39.984844 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:58:39.984844 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:58:39.984844 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:58:39.984844 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:58:39.985697 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:58:39.985697 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 24 05:58:39.985697 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:58:39.985697 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:58:39.985697 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:58:39.985697 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:58:39.985697 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:58:39.985697 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:58:39.985697 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:58:39.985697 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:58:39.985697 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 05:58:39.985697 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 05:58:39.985697 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 05:58:39.985697 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 24 05:58:39.985697 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 524/1046] 10.222.0.62 () {56 vars in 1105 bytes} [Wed Aug 24 05:58:39 2022] GET /identity/v3/services/image => generated 86 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 05:58:39.990405 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907b3415-26f6-4fae-a551-3a07be90b615 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:40.071687 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe64c8d6-9ec3-4132-a071-988be1687865 None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:40.093011 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907b3415-26f6-4fae-a551-3a07be90b615 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:40.105926 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:40.108837 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f71967-5f50-4563-9a0d-af7ef852165b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:40.109617 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f71967-5f50-4563-9a0d-af7ef852165b None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:40.110046 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f71967-5f50-4563-9a0d-af7ef852165b None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:40.110573 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f71967-5f50-4563-9a0d-af7ef852165b None admin] PATH_INFO: `/v3/projects` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:40.111262 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f71967-5f50-4563-9a0d-af7ef852165b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:58:40.111567 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f71967-5f50-4563-9a0d-af7ef852165b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:40.113024 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f71967-5f50-4563-9a0d-af7ef852165b None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:40.126331 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 523/1047] 10.222.0.62 () {56 vars in 1148 bytes} [Wed Aug 24 05:58:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:58:40.128369 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17631fc8-28c3-4a5f-899e-2a03583e4fde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:40.129246 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17631fc8-28c3-4a5f-899e-2a03583e4fde None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:40.129720 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17631fc8-28c3-4a5f-899e-2a03583e4fde None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:40.130186 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17631fc8-28c3-4a5f-899e-2a03583e4fde None admin] PATH_INFO: `/v3/services` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:40.130996 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17631fc8-28c3-4a5f-899e-2a03583e4fde None admin] RBAC: Adding query filter params (name=image) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:58:40.131528 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17631fc8-28c3-4a5f-899e-2a03583e4fde None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:40.132358 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f71967-5f50-4563-9a0d-af7ef852165b None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:40.132912 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17631fc8-28c3-4a5f-899e-2a03583e4fde None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:40.138125 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 525/1048] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Aug 24 05:58:39 2022] GET /identity/v3/services?name=image => generated 122 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:58:40.143854 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17631fc8-28c3-4a5f-899e-2a03583e4fde None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:40.153860 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:40.233913 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f71967-5f50-4563-9a0d-af7ef852165b None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:40.244420 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17631fc8-28c3-4a5f-899e-2a03583e4fde None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:40.267177 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cdca1f-5654-4953-8095-f5a5e3d3c159 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:40.267771 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cdca1f-5654-4953-8095-f5a5e3d3c159 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:40.268049 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cdca1f-5654-4953-8095-f5a5e3d3c159 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:40.268323 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cdca1f-5654-4953-8095-f5a5e3d3c159 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:40.269065 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1cdca1f-5654-4953-8095-f5a5e3d3c159 None admin] RBAC: Adding query filter params (role.id=2499f7e98f3e498ab329637f00bb2da4, scope.project.id=181f4762037e454489d9552cc603179c, user.id=fb6d3743340241f98cacb9e0d063298c) {{(pid=81851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:58:40.269374 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1cdca1f-5654-4953-8095-f5a5e3d3c159 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:40.270802 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1cdca1f-5654-4953-8095-f5a5e3d3c159 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:40.276153 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 524/1049] 10.222.0.62 () {56 vars in 1374 bytes} [Wed Aug 24 05:58:40 2022] GET /identity/v3/role_assignments?user.id=fb6d3743340241f98cacb9e0d063298c&scope.project.id=181f4762037e454489d9552cc603179c&role.id=2499f7e98f3e498ab329637f00bb2da4 => generated 259 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:58:40.277328 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f0e36a-9c9e-42ea-b0b6-01bc4bc21434 None None] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:40.277606 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f0e36a-9c9e-42ea-b0b6-01bc4bc21434 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:40.277876 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f0e36a-9c9e-42ea-b0b6-01bc4bc21434 None None] PATH_INFO: `/` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:40.279083 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 525/1050] 10.222.0.62 () {54 vars in 927 bytes} [Wed Aug 24 05:58:40 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:58:40.284828 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2c57d1-9650-4dc3-be54-2999bd16145a None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:40.285116 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2c57d1-9650-4dc3-be54-2999bd16145a None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:40.285397 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2c57d1-9650-4dc3-be54-2999bd16145a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:40.286682 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd129ab2-0558-4bde-9189-e23193f81e61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:40.287311 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd129ab2-0558-4bde-9189-e23193f81e61 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:40.287585 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd129ab2-0558-4bde-9189-e23193f81e61 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:40.287851 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd129ab2-0558-4bde-9189-e23193f81e61 None admin] PATH_INFO: `/v3/services` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:40.288486 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd129ab2-0558-4bde-9189-e23193f81e61 None admin] RBAC: Adding query filter params (type=image) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:58:40.288819 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd129ab2-0558-4bde-9189-e23193f81e61 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:40.290220 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd129ab2-0558-4bde-9189-e23193f81e61 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:40.298419 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 526/1051] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Aug 24 05:58:40 2022] GET /identity/v3/services?type=image => generated 350 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:58:40.305306 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-ab2c57d1-9650-4dc3-be54-2999bd16145a None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:58:40.306543 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-ab2c57d1-9650-4dc3-be54-2999bd16145a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:58:40.306762 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd129ab2-0558-4bde-9189-e23193f81e61 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:40.358945 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:40.381675 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 526/1052] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Aug 24 05:58:40 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 05:58:40.438306 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd129ab2-0558-4bde-9189-e23193f81e61 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:40.472599 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa0afbe-7498-45b2-8c83-7b008d200518 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:40.473225 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa0afbe-7498-45b2-8c83-7b008d200518 None admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:40.473471 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa0afbe-7498-45b2-8c83-7b008d200518 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:40.473739 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa0afbe-7498-45b2-8c83-7b008d200518 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:40.474376 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa0afbe-7498-45b2-8c83-7b008d200518 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:40.475726 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa0afbe-7498-45b2-8c83-7b008d200518 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:40.586760 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 527/1053] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Aug 24 05:58:40 2022] POST /identity/v3/endpoints => generated 327 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 05:58:40.655670 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2c57d1-9650-4dc3-be54-2999bd16145a None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:40.671578 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:40.773198 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2c57d1-9650-4dc3-be54-2999bd16145a None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:40.816084 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67866460-1e1e-4ee8-a605-748e552be6bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:40.816751 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67866460-1e1e-4ee8-a605-748e552be6bf None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:40.817030 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67866460-1e1e-4ee8-a605-748e552be6bf None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:40.817300 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67866460-1e1e-4ee8-a605-748e552be6bf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:40.828203 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67866460-1e1e-4ee8-a605-748e552be6bf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:58:40.832894 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application [None req-67866460-1e1e-4ee8-a605-748e552be6bf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 05:58:40.832894 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:58:40.832894 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:58:40.832894 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:58:40.832894 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:58:40.832894 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:58:40.832894 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:58:40.832894 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:58:40.832894 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:58:40.832894 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:58:40.832894 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:58:40.832894 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:58:40.832894 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:58:40.832894 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:58:40.832894 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:58:40.832894 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:58:40.833820 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:58:40.833820 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:58:40.833820 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:58:40.833820 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:58:40.833820 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:58:40.833820 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:58:40.833820 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:58:40.833820 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:58:40.833820 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:58:40.833820 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:58:40.833820 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:58:40.833820 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application with Lock( Aug 24 05:58:40.833820 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:58:40.833820 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:58:40.833820 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:58:40.833820 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:58:40.835035 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:58:40.835035 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:58:40.835035 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:58:40.835035 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:58:40.835035 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:58:40.835035 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:58:40.835035 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:58:40.835035 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:58:40.835035 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:58:40.835035 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:58:40.835035 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 05:58:40.835035 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Aug 24 05:58:40.835035 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 527/1054] 10.222.0.62 () {56 vars in 1096 bytes} [Wed Aug 24 05:58:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:58:40.840114 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa0afbe-7498-45b2-8c83-7b008d200518 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:40.943873 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:40.950620 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa0afbe-7498-45b2-8c83-7b008d200518 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:40.984479 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed5893b-a829-4c2e-abfb-dc789922e075 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:40.985108 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed5893b-a829-4c2e-abfb-dc789922e075 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:40.985386 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed5893b-a829-4c2e-abfb-dc789922e075 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:40.985654 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed5893b-a829-4c2e-abfb-dc789922e075 None admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:40.986306 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed5893b-a829-4c2e-abfb-dc789922e075 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:58:40.986608 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed5893b-a829-4c2e-abfb-dc789922e075 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:40.987966 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed5893b-a829-4c2e-abfb-dc789922e075 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:41.001167 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 528/1055] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Aug 24 05:58:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:58:41.004005 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67866460-1e1e-4ee8-a605-748e552be6bf None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:41.073261 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:41.105914 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67866460-1e1e-4ee8-a605-748e552be6bf None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:41.140426 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba39cc6-0849-414a-bb41-4e2bc9277848 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:41.141079 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba39cc6-0849-414a-bb41-4e2bc9277848 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:41.141353 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba39cc6-0849-414a-bb41-4e2bc9277848 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:41.141620 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba39cc6-0849-414a-bb41-4e2bc9277848 None admin] PATH_INFO: `/v3/users/027c81b97b8a4e82a55477f1e5c555be` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:41.155308 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba39cc6-0849-414a-bb41-4e2bc9277848 None admin] RBAC: Authorizing `identity:get_user(user_id=027c81b97b8a4e82a55477f1e5c555be)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:41.156795 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba39cc6-0849-414a-bb41-4e2bc9277848 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:41.163269 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 528/1056] 10.222.0.62 () {56 vars in 1177 bytes} [Wed Aug 24 05:58:41 2022] GET /identity/v3/users/027c81b97b8a4e82a55477f1e5c555be => generated 285 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:58:41.164717 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed5893b-a829-4c2e-abfb-dc789922e075 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:41.265936 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed5893b-a829-4c2e-abfb-dc789922e075 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:41.299571 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16529b9f-692c-4e15-9d8d-cbb2351e080b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:41.300174 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16529b9f-692c-4e15-9d8d-cbb2351e080b None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:41.300449 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16529b9f-692c-4e15-9d8d-cbb2351e080b None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:41.300760 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16529b9f-692c-4e15-9d8d-cbb2351e080b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:41.326935 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:41.370289 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16529b9f-692c-4e15-9d8d-cbb2351e080b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:41.371747 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16529b9f-692c-4e15-9d8d-cbb2351e080b None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:41.471455 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 529/1057] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Aug 24 05:58:41 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 24 05:58:41.473321 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba39cc6-0849-414a-bb41-4e2bc9277848 None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:41.573729 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba39cc6-0849-414a-bb41-4e2bc9277848 None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:41.599880 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:41.608917 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210c618c-8cc2-4b8d-b76c-5eda25a59aeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:41.609544 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210c618c-8cc2-4b8d-b76c-5eda25a59aeb None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:41.609822 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210c618c-8cc2-4b8d-b76c-5eda25a59aeb None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:41.610092 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210c618c-8cc2-4b8d-b76c-5eda25a59aeb None admin] PATH_INFO: `/v3/projects/b68006cd169244c5b544c7e16d1e759d` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:41.611197 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210c618c-8cc2-4b8d-b76c-5eda25a59aeb None admin] RBAC: Authorizing `identity:get_project(project_id=b68006cd169244c5b544c7e16d1e759d)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:41.612659 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210c618c-8cc2-4b8d-b76c-5eda25a59aeb None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:41.619313 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 529/1058] 10.222.0.62 () {56 vars in 1186 bytes} [Wed Aug 24 05:58:41 2022] GET /identity/v3/projects/b68006cd169244c5b544c7e16d1e759d => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:58:41.621030 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16529b9f-692c-4e15-9d8d-cbb2351e080b None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:41.680126 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:41.731444 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16529b9f-692c-4e15-9d8d-cbb2351e080b None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:41.765747 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fe14df-635e-4f42-ad75-486395984bad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:41.766352 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fe14df-635e-4f42-ad75-486395984bad None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:41.766647 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fe14df-635e-4f42-ad75-486395984bad None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:41.766917 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fe14df-635e-4f42-ad75-486395984bad None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:41.767619 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fe14df-635e-4f42-ad75-486395984bad None admin] RBAC: Adding query filter params (role.id=49e47cd157d743f6905f1db07ec878a7, scope.project.id=b68006cd169244c5b544c7e16d1e759d, user.id=027c81b97b8a4e82a55477f1e5c555be) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:58:41.767917 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fe14df-635e-4f42-ad75-486395984bad None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:41.769430 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fe14df-635e-4f42-ad75-486395984bad None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:41.776651 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 530/1059] 10.222.0.62 () {56 vars in 1374 bytes} [Wed Aug 24 05:58:41 2022] GET /identity/v3/role_assignments?user.id=027c81b97b8a4e82a55477f1e5c555be&scope.project.id=b68006cd169244c5b544c7e16d1e759d&role.id=49e47cd157d743f6905f1db07ec878a7 => generated 259 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:58:42.180383 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a38225d-d5c9-4c56-bdb4-129b59bb9dbd None None] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:42.180942 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a38225d-d5c9-4c56-bdb4-129b59bb9dbd None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:42.180942 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a38225d-d5c9-4c56-bdb4-129b59bb9dbd None None] PATH_INFO: `/` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:42.182478 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 530/1060] 10.222.0.62 () {54 vars in 927 bytes} [Wed Aug 24 05:58:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:58:42.187973 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9425eb4f-9f5b-4681-9bbb-f060dc06feaa None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:42.188324 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9425eb4f-9f5b-4681-9bbb-f060dc06feaa None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:42.188694 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9425eb4f-9f5b-4681-9bbb-f060dc06feaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:42.207321 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-9425eb4f-9f5b-4681-9bbb-f060dc06feaa None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:58:42.208556 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-9425eb4f-9f5b-4681-9bbb-f060dc06feaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:58:42.280533 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 531/1061] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Aug 24 05:58:42 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 05:58:42.508726 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a200b8-d3d1-4b34-b4c3-adf6b99eccc4 None None] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:42.509020 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a200b8-d3d1-4b34-b4c3-adf6b99eccc4 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:42.509206 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a200b8-d3d1-4b34-b4c3-adf6b99eccc4 None None] PATH_INFO: `/` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:42.510698 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 531/1062] 10.222.0.62 () {54 vars in 927 bytes} [Wed Aug 24 05:58:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:58:42.516180 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bacdd2-bad0-4729-8066-f96e8ababe9f None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:42.516473 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bacdd2-bad0-4729-8066-f96e8ababe9f None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:42.516785 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bacdd2-bad0-4729-8066-f96e8ababe9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:42.536063 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-e0bacdd2-bad0-4729-8066-f96e8ababe9f None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:58:42.537271 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-e0bacdd2-bad0-4729-8066-f96e8ababe9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:58:42.545983 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a200b8-d3d1-4b34-b4c3-adf6b99eccc4 None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:42.588258 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:42.609399 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 532/1063] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Aug 24 05:58:42 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 05:58:42.657411 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a200b8-d3d1-4b34-b4c3-adf6b99eccc4 None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:42.694439 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a075862-5797-466a-b6ce-5a597422552b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:42.695066 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a075862-5797-466a-b6ce-5a597422552b None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:42.695342 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a075862-5797-466a-b6ce-5a597422552b None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:42.695603 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a075862-5797-466a-b6ce-5a597422552b None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:42.709214 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a075862-5797-466a-b6ce-5a597422552b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:58:42.713863 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application [None req-6a075862-5797-466a-b6ce-5a597422552b None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 05:58:42.713863 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:58:42.713863 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:58:42.713863 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:58:42.713863 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:58:42.713863 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:58:42.713863 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:58:42.713863 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:58:42.713863 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:58:42.713863 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:58:42.713863 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:58:42.713863 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:58:42.713863 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:58:42.713863 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:58:42.713863 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:58:42.713863 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:58:42.714810 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:58:42.714810 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:58:42.714810 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:58:42.714810 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:58:42.714810 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:58:42.714810 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:58:42.714810 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:58:42.714810 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:58:42.714810 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:58:42.714810 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:58:42.714810 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:58:42.714810 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application with Lock( Aug 24 05:58:42.714810 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:58:42.714810 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:58:42.714810 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:58:42.714810 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:58:42.715793 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:58:42.715793 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:58:42.715793 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:58:42.715793 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:58:42.715793 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:58:42.715793 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:58:42.715793 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:58:42.715793 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:58:42.715793 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:58:42.715793 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:58:42.715793 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 05:58:42.715793 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Aug 24 05:58:42.715793 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 532/1064] 10.222.0.62 () {56 vars in 1120 bytes} [Wed Aug 24 05:58:42 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 05:58:42.721052 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bacdd2-bad0-4729-8066-f96e8ababe9f None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:42.821271 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bacdd2-bad0-4729-8066-f96e8ababe9f None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:42.857276 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893205d0-06ed-4bd4-9601-0808dc76fc44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:42.857880 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893205d0-06ed-4bd4-9601-0808dc76fc44 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:42.858154 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893205d0-06ed-4bd4-9601-0808dc76fc44 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:42.858423 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893205d0-06ed-4bd4-9601-0808dc76fc44 None admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:42.859071 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893205d0-06ed-4bd4-9601-0808dc76fc44 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:58:42.859370 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893205d0-06ed-4bd4-9601-0808dc76fc44 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:42.860746 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893205d0-06ed-4bd4-9601-0808dc76fc44 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:42.868541 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 533/1065] 10.222.0.62 () {56 vars in 1115 bytes} [Wed Aug 24 05:58:42 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:58:42.874718 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a075862-5797-466a-b6ce-5a597422552b None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:42.876928 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893205d0-06ed-4bd4-9601-0808dc76fc44 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:42.923702 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:42.931120 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:42.976086 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a075862-5797-466a-b6ce-5a597422552b None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:42.992228 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893205d0-06ed-4bd4-9601-0808dc76fc44 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:43.009776 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a726fb-2f19-45b2-bb23-e272f902e0fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:43.010399 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a726fb-2f19-45b2-bb23-e272f902e0fc None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:43.010675 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a726fb-2f19-45b2-bb23-e272f902e0fc None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:43.010943 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a726fb-2f19-45b2-bb23-e272f902e0fc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:43.021951 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a726fb-2f19-45b2-bb23-e272f902e0fc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:43.023382 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a726fb-2f19-45b2-bb23-e272f902e0fc None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:43.027960 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8818a65b-fb98-4f27-8132-0b93fe3df7b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:43.028605 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8818a65b-fb98-4f27-8132-0b93fe3df7b1 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:43.028885 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8818a65b-fb98-4f27-8132-0b93fe3df7b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:43.029155 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8818a65b-fb98-4f27-8132-0b93fe3df7b1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:43.037217 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application [None req-60a726fb-2f19-45b2-bb23-e272f902e0fc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:58:43.037217 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:58:43.037217 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:58:43.037217 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:58:43.037217 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:58:43.037217 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:58:43.037217 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:58:43.037217 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:58:43.037217 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:58:43.037217 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:58:43.037217 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:58:43.037217 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:58:43.037217 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 05:58:43.037217 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 05:58:43.037217 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 05:58:43.037217 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 05:58:43.038148 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:58:43.038148 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:58:43.038148 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:58:43.038148 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:58:43.038148 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:58:43.038148 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:58:43.038148 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:58:43.038148 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application with Lock( Aug 24 05:58:43.038148 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:58:43.038148 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:58:43.038148 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:58:43.038148 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:58:43.038148 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:58:43.038148 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:58:43.038148 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:58:43.038148 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:58:43.038148 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 05:58:43.044294 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 05:58:43.044294 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:58:43.044294 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Aug 24 05:58:43.044294 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 533/1066] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Aug 24 05:58:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:58:43.044569 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a726fb-2f19-45b2-bb23-e272f902e0fc None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:43.107450 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8818a65b-fb98-4f27-8132-0b93fe3df7b1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:43.108849 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8818a65b-fb98-4f27-8132-0b93fe3df7b1 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:43.145048 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a726fb-2f19-45b2-bb23-e272f902e0fc None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:43.152063 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:43.155232 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:43.180892 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82c6089-fab1-4b18-aa91-6c7791c38246 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:43.181589 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82c6089-fab1-4b18-aa91-6c7791c38246 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:43.181779 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82c6089-fab1-4b18-aa91-6c7791c38246 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:43.182027 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82c6089-fab1-4b18-aa91-6c7791c38246 None admin] PATH_INFO: `/v3/domains` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:43.182662 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82c6089-fab1-4b18-aa91-6c7791c38246 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:58:43.182949 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82c6089-fab1-4b18-aa91-6c7791c38246 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:43.184907 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82c6089-fab1-4b18-aa91-6c7791c38246 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:43.197296 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 534/1067] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Aug 24 05:58:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:58:43.202885 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82c6089-fab1-4b18-aa91-6c7791c38246 None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:43.216011 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 534/1068] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Aug 24 05:58:42 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 24 05:58:43.221687 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8818a65b-fb98-4f27-8132-0b93fe3df7b1 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:43.311290 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82c6089-fab1-4b18-aa91-6c7791c38246 None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:43.326761 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8818a65b-fb98-4f27-8132-0b93fe3df7b1 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:43.350875 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc93dcf-2ee1-4693-99de-4deed54389dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:43.351509 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc93dcf-2ee1-4693-99de-4deed54389dd None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:43.351789 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc93dcf-2ee1-4693-99de-4deed54389dd None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:43.352061 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc93dcf-2ee1-4693-99de-4deed54389dd None admin] PATH_INFO: `/v3/users/nova` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:43.360343 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc93dcf-2ee1-4693-99de-4deed54389dd None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:43.361777 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc93dcf-2ee1-4693-99de-4deed54389dd None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:43.373388 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application [None req-0bc93dcf-2ee1-4693-99de-4deed54389dd None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 05:58:43.373388 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:58:43.373388 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:58:43.373388 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:58:43.373388 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:58:43.373388 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:58:43.373388 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:58:43.373388 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:58:43.373388 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:58:43.373388 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:58:43.373388 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:58:43.373388 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:58:43.373388 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 05:58:43.373388 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 05:58:43.373388 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 05:58:43.373388 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 05:58:43.374344 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:58:43.374344 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:58:43.374344 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 05:58:43.374344 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 05:58:43.374344 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 05:58:43.374344 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 05:58:43.374344 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:58:43.374344 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:58:43.374344 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:58:43.374344 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:58:43.374344 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:58:43.374344 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application with Lock( Aug 24 05:58:43.374344 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:58:43.374344 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:58:43.374344 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:58:43.374344 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:58:43.374344 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:58:43.374344 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:58:43.375750 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:58:43.375750 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:58:43.375750 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 05:58:43.375750 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 05:58:43.375750 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 05:58:43.375750 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 05:58:43.375750 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 05:58:43.375750 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 05:58:43.375750 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 05:58:43.375750 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Aug 24 05:58:43.375750 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 535/1069] 10.222.0.62 () {56 vars in 1093 bytes} [Wed Aug 24 05:58:43 2022] GET /identity/v3/users/nova => generated 82 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 05:58:43.378018 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca03fe6d-ffa0-4741-a8e5-1d77e31cfcb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:43.378812 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca03fe6d-ffa0-4741-a8e5-1d77e31cfcb2 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:43.379335 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca03fe6d-ffa0-4741-a8e5-1d77e31cfcb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:43.379611 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca03fe6d-ffa0-4741-a8e5-1d77e31cfcb2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:43.380409 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc93dcf-2ee1-4693-99de-4deed54389dd None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:43.389790 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca03fe6d-ffa0-4741-a8e5-1d77e31cfcb2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:43.391239 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca03fe6d-ffa0-4741-a8e5-1d77e31cfcb2 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:43.393767 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:43.406226 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-ca03fe6d-ffa0-4741-a8e5-1d77e31cfcb2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:58:43.406226 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:58:43.406226 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:58:43.406226 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:58:43.406226 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:58:43.406226 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:58:43.406226 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:58:43.406226 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:58:43.406226 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:58:43.406226 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:58:43.406226 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:58:43.406226 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:58:43.406226 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 05:58:43.406226 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 05:58:43.406226 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 05:58:43.406226 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 05:58:43.407175 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:58:43.407175 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:58:43.407175 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:58:43.407175 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:58:43.407175 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:58:43.407175 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:58:43.407175 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:58:43.407175 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 24 05:58:43.407175 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:58:43.407175 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:58:43.407175 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:58:43.407175 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:58:43.407175 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:58:43.407175 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:58:43.407175 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:58:43.407175 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:58:43.407175 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 05:58:43.408116 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 05:58:43.408116 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:58:43.408116 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 24 05:58:43.408116 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 535/1070] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Aug 24 05:58:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 05:58:43.413010 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca03fe6d-ffa0-4741-a8e5-1d77e31cfcb2 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:43.417642 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:43.491483 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc93dcf-2ee1-4693-99de-4deed54389dd None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:43.520997 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca03fe6d-ffa0-4741-a8e5-1d77e31cfcb2 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:43.527818 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916f138e-285c-4490-9bc1-e9cd52b0315e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:43.529416 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916f138e-285c-4490-9bc1-e9cd52b0315e None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:43.529702 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916f138e-285c-4490-9bc1-e9cd52b0315e None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:43.529976 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916f138e-285c-4490-9bc1-e9cd52b0315e None admin] PATH_INFO: `/v3/users` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:43.530705 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916f138e-285c-4490-9bc1-e9cd52b0315e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:58:43.531376 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916f138e-285c-4490-9bc1-e9cd52b0315e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:43.533172 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916f138e-285c-4490-9bc1-e9cd52b0315e None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:43.556275 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 536/1071] 10.222.0.62 () {56 vars in 1133 bytes} [Wed Aug 24 05:58:43 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:58:43.557706 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916f138e-285c-4490-9bc1-e9cd52b0315e None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:43.562937 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966c4c5d-df83-4412-81bd-bd5a21294905 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:43.563773 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966c4c5d-df83-4412-81bd-bd5a21294905 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:43.564228 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966c4c5d-df83-4412-81bd-bd5a21294905 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:43.564853 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966c4c5d-df83-4412-81bd-bd5a21294905 None admin] PATH_INFO: `/v3/domains` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:43.565664 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966c4c5d-df83-4412-81bd-bd5a21294905 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:58:43.566258 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966c4c5d-df83-4412-81bd-bd5a21294905 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:43.567615 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966c4c5d-df83-4412-81bd-bd5a21294905 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:43.584841 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 536/1072] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Aug 24 05:58:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:58:43.649685 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:43.667789 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916f138e-285c-4490-9bc1-e9cd52b0315e None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:43.674434 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfbe03f-d9fb-4d18-9903-c0a48c906a44 None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:43.675065 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfbe03f-d9fb-4d18-9903-c0a48c906a44 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:43.675567 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfbe03f-d9fb-4d18-9903-c0a48c906a44 None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:43.677491 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 537/1073] 10.222.0.62 () {54 vars in 927 bytes} [Wed Aug 24 05:58:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 05:58:43.682752 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0263e28-c855-4976-b9dc-0bea4919aac4 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:43.683033 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0263e28-c855-4976-b9dc-0bea4919aac4 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:43.683311 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0263e28-c855-4976-b9dc-0bea4919aac4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:43.701775 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-a0263e28-c855-4976-b9dc-0bea4919aac4 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:58:43.703016 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-a0263e28-c855-4976-b9dc-0bea4919aac4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:58:43.718898 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c95f502-76f5-4f9c-b186-8d6b35e94f1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:43.719545 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c95f502-76f5-4f9c-b186-8d6b35e94f1b None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:43.719826 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c95f502-76f5-4f9c-b186-8d6b35e94f1b None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:43.720100 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c95f502-76f5-4f9c-b186-8d6b35e94f1b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:43.732879 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c95f502-76f5-4f9c-b186-8d6b35e94f1b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:43.734604 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c95f502-76f5-4f9c-b186-8d6b35e94f1b None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:43.751379 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application [None req-2c95f502-76f5-4f9c-b186-8d6b35e94f1b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:58:43.751379 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:58:43.751379 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:58:43.751379 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:58:43.751379 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:58:43.751379 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:58:43.751379 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:58:43.751379 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:58:43.751379 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:58:43.751379 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:58:43.751379 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:58:43.751379 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:58:43.751379 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 05:58:43.751379 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 05:58:43.751379 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 05:58:43.751379 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 05:58:43.752563 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:58:43.752563 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:58:43.752563 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:58:43.752563 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:58:43.752563 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:58:43.752563 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:58:43.752563 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:58:43.752563 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application with Lock( Aug 24 05:58:43.752563 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:58:43.752563 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:58:43.752563 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:58:43.752563 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:58:43.752563 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:58:43.752563 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:58:43.752563 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:58:43.752563 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:58:43.752563 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 05:58:43.753472 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 05:58:43.753472 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:58:43.753472 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Aug 24 05:58:43.753472 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 537/1074] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Aug 24 05:58:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 05:58:43.759665 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c95f502-76f5-4f9c-b186-8d6b35e94f1b None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:43.767490 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:43.788568 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 538/1075] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Aug 24 05:58:43 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 05:58:43.870165 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c95f502-76f5-4f9c-b186-8d6b35e94f1b None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:43.905794 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d8e590-1593-4b7f-9952-d9307383a2c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:43.906328 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d8e590-1593-4b7f-9952-d9307383a2c3 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:43.906598 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d8e590-1593-4b7f-9952-d9307383a2c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:43.906863 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d8e590-1593-4b7f-9952-d9307383a2c3 None admin] PATH_INFO: `/v3/domains` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:43.907489 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d8e590-1593-4b7f-9952-d9307383a2c3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:58:43.907785 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d8e590-1593-4b7f-9952-d9307383a2c3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:43.909167 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d8e590-1593-4b7f-9952-d9307383a2c3 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:43.911890 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:43.921450 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 538/1076] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Aug 24 05:58:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:58:43.927221 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0263e28-c855-4976-b9dc-0bea4919aac4 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:44.024342 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0263e28-c855-4976-b9dc-0bea4919aac4 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:44.058327 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca38101-1347-4e12-b0f5-ee5a7e9aea93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:44.059177 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca38101-1347-4e12-b0f5-ee5a7e9aea93 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:44.059450 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca38101-1347-4e12-b0f5-ee5a7e9aea93 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:44.059588 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d8e590-1593-4b7f-9952-d9307383a2c3 None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:44.059717 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca38101-1347-4e12-b0f5-ee5a7e9aea93 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:44.066411 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca38101-1347-4e12-b0f5-ee5a7e9aea93 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:44.067852 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca38101-1347-4e12-b0f5-ee5a7e9aea93 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:44.079784 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-7ca38101-1347-4e12-b0f5-ee5a7e9aea93 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 05:58:44.079784 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:58:44.079784 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:58:44.079784 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:58:44.079784 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:58:44.079784 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:58:44.079784 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:58:44.079784 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:58:44.079784 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:58:44.079784 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:58:44.079784 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:58:44.079784 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:58:44.079784 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 05:58:44.079784 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 05:58:44.079784 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 05:58:44.080698 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 05:58:44.080698 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:58:44.080698 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:58:44.080698 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:58:44.080698 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:58:44.080698 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:58:44.080698 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:58:44.080698 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:58:44.080698 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 24 05:58:44.080698 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:58:44.080698 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:58:44.080698 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:58:44.080698 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:58:44.080698 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:58:44.080698 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:58:44.080698 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:58:44.080698 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:58:44.081581 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 05:58:44.081581 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 05:58:44.081581 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 05:58:44.081581 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 05:58:44.081581 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 05:58:44.081581 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 05:58:44.081581 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 05:58:44.081581 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 24 05:58:44.081581 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 539/1077] 10.222.0.62 () {56 vars in 1111 bytes} [Wed Aug 24 05:58:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:58:44.086527 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca38101-1347-4e12-b0f5-ee5a7e9aea93 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:44.131728 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:44.159045 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d8e590-1593-4b7f-9952-d9307383a2c3 None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:44.183491 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca38101-1347-4e12-b0f5-ee5a7e9aea93 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:44.193013 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417bf2db-1ee3-4e71-afd3-06519daac955 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:44.193638 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417bf2db-1ee3-4e71-afd3-06519daac955 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:44.193917 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417bf2db-1ee3-4e71-afd3-06519daac955 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:44.194194 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417bf2db-1ee3-4e71-afd3-06519daac955 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:44.204716 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417bf2db-1ee3-4e71-afd3-06519daac955 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:58:44.209404 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application [None req-417bf2db-1ee3-4e71-afd3-06519daac955 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 05:58:44.209404 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:58:44.209404 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:58:44.209404 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:58:44.209404 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:58:44.209404 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:58:44.209404 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:58:44.209404 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:58:44.209404 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:58:44.209404 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:58:44.209404 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:58:44.209404 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:58:44.209404 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:58:44.209404 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:58:44.209404 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:58:44.209404 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:58:44.210508 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:58:44.210508 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:58:44.210508 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:58:44.210508 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:58:44.210508 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:58:44.210508 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:58:44.210508 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:58:44.210508 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:58:44.210508 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:58:44.210508 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:58:44.210508 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:58:44.210508 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application with Lock( Aug 24 05:58:44.210508 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:58:44.210508 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:58:44.210508 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:58:44.210508 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:58:44.211557 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:58:44.211557 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:58:44.211557 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:58:44.211557 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:58:44.211557 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:58:44.211557 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:58:44.211557 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:58:44.211557 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:58:44.211557 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:58:44.211557 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:58:44.211557 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 05:58:44.211557 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Aug 24 05:58:44.211557 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 539/1078] 10.222.0.62 () {56 vars in 1096 bytes} [Wed Aug 24 05:58:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:58:44.216521 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417bf2db-1ee3-4e71-afd3-06519daac955 None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:44.220215 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389bf3c6-c002-46aa-ad94-c43e373d17c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:44.220850 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389bf3c6-c002-46aa-ad94-c43e373d17c0 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:44.221126 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389bf3c6-c002-46aa-ad94-c43e373d17c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:44.221396 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389bf3c6-c002-46aa-ad94-c43e373d17c0 None admin] PATH_INFO: `/v3/projects` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:44.222081 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389bf3c6-c002-46aa-ad94-c43e373d17c0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:58:44.222383 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389bf3c6-c002-46aa-ad94-c43e373d17c0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:44.223800 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389bf3c6-c002-46aa-ad94-c43e373d17c0 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:44.236038 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 540/1079] 10.222.0.62 () {56 vars in 1148 bytes} [Wed Aug 24 05:58:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:58:44.241616 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389bf3c6-c002-46aa-ad94-c43e373d17c0 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:44.273784 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:44.316937 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417bf2db-1ee3-4e71-afd3-06519daac955 None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:44.341837 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389bf3c6-c002-46aa-ad94-c43e373d17c0 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:44.348844 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:44.350166 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47a4dbd-f746-4d14-a5a2-7e301bfeb9f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:44.350764 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47a4dbd-f746-4d14-a5a2-7e301bfeb9f5 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:44.351040 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47a4dbd-f746-4d14-a5a2-7e301bfeb9f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:44.351306 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47a4dbd-f746-4d14-a5a2-7e301bfeb9f5 None admin] PATH_INFO: `/v3/roles` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:44.351950 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47a4dbd-f746-4d14-a5a2-7e301bfeb9f5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:58:44.352246 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47a4dbd-f746-4d14-a5a2-7e301bfeb9f5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:44.353649 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47a4dbd-f746-4d14-a5a2-7e301bfeb9f5 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:44.361930 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 540/1080] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Aug 24 05:58:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:58:44.367680 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47a4dbd-f746-4d14-a5a2-7e301bfeb9f5 None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:44.375676 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595a8414-15a5-401f-8979-3416d8ddb1bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:44.376331 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595a8414-15a5-401f-8979-3416d8ddb1bc None admin] REQUEST_METHOD: `PUT` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:44.376639 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595a8414-15a5-401f-8979-3416d8ddb1bc None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:44.376916 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595a8414-15a5-401f-8979-3416d8ddb1bc None admin] PATH_INFO: `/v3/projects/181f4762037e454489d9552cc603179c/users/fb6d3743340241f98cacb9e0d063298c/roles/2499f7e98f3e498ab329637f00bb2da4` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:44.379044 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595a8414-15a5-401f-8979-3416d8ddb1bc None admin] RBAC: Authorizing `identity:create_grant(project_id=181f4762037e454489d9552cc603179c, user_id=fb6d3743340241f98cacb9e0d063298c, role_id=2499f7e98f3e498ab329637f00bb2da4)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:44.380462 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595a8414-15a5-401f-8979-3416d8ddb1bc None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:44.463200 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47a4dbd-f746-4d14-a5a2-7e301bfeb9f5 None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:44.479774 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:44.498563 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5af4d1b-a603-4ad8-a261-609d1fd2967f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:44.499182 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5af4d1b-a603-4ad8-a261-609d1fd2967f None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:44.499468 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5af4d1b-a603-4ad8-a261-609d1fd2967f None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:44.499739 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5af4d1b-a603-4ad8-a261-609d1fd2967f None admin] PATH_INFO: `/v3/users/027c81b97b8a4e82a55477f1e5c555be` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:44.500861 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 541/1081] 10.222.0.62 () {58 vars in 1439 bytes} [Wed Aug 24 05:58:44 2022] PUT /identity/v3/projects/181f4762037e454489d9552cc603179c/users/fb6d3743340241f98cacb9e0d063298c/roles/2499f7e98f3e498ab329637f00bb2da4 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 05:58:44.509057 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5af4d1b-a603-4ad8-a261-609d1fd2967f None admin] RBAC: Authorizing `identity:get_user(user_id=027c81b97b8a4e82a55477f1e5c555be)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:44.509057 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5af4d1b-a603-4ad8-a261-609d1fd2967f None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:44.509057 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 541/1082] 10.222.0.62 () {56 vars in 1177 bytes} [Wed Aug 24 05:58:44 2022] GET /identity/v3/users/027c81b97b8a4e82a55477f1e5c555be => generated 285 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:58:44.510033 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595a8414-15a5-401f-8979-3416d8ddb1bc None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:44.619349 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595a8414-15a5-401f-8979-3416d8ddb1bc None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:44.661153 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b82264c-2aca-42f7-9690-f8a1be849631 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:44.661809 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b82264c-2aca-42f7-9690-f8a1be849631 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:44.662083 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b82264c-2aca-42f7-9690-f8a1be849631 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:44.662350 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b82264c-2aca-42f7-9690-f8a1be849631 None admin] PATH_INFO: `/v3/projects/b68006cd169244c5b544c7e16d1e759d` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:44.663476 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b82264c-2aca-42f7-9690-f8a1be849631 None admin] RBAC: Authorizing `identity:get_project(project_id=b68006cd169244c5b544c7e16d1e759d)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:44.664928 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b82264c-2aca-42f7-9690-f8a1be849631 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:44.668414 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 542/1083] 10.222.0.62 () {56 vars in 1186 bytes} [Wed Aug 24 05:58:44 2022] GET /identity/v3/projects/b68006cd169244c5b544c7e16d1e759d => generated 311 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:58:44.673696 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5af4d1b-a603-4ad8-a261-609d1fd2967f None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:44.782396 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5af4d1b-a603-4ad8-a261-609d1fd2967f None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:44.817353 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56877bcf-a409-40b6-a833-7741ddccb259 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:44.818009 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56877bcf-a409-40b6-a833-7741ddccb259 None admin] REQUEST_METHOD: `PUT` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:44.818284 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56877bcf-a409-40b6-a833-7741ddccb259 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:44.818554 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56877bcf-a409-40b6-a833-7741ddccb259 None admin] PATH_INFO: `/v3/projects/b68006cd169244c5b544c7e16d1e759d/users/027c81b97b8a4e82a55477f1e5c555be/roles/49e47cd157d743f6905f1db07ec878a7` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:44.820445 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56877bcf-a409-40b6-a833-7741ddccb259 None admin] RBAC: Authorizing `identity:create_grant(project_id=b68006cd169244c5b544c7e16d1e759d, user_id=027c81b97b8a4e82a55477f1e5c555be, role_id=49e47cd157d743f6905f1db07ec878a7)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:44.821900 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56877bcf-a409-40b6-a833-7741ddccb259 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:44.889077 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 542/1084] 10.222.0.62 () {58 vars in 1439 bytes} [Wed Aug 24 05:58:44 2022] PUT /identity/v3/projects/b68006cd169244c5b544c7e16d1e759d/users/027c81b97b8a4e82a55477f1e5c555be/roles/49e47cd157d743f6905f1db07ec878a7 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:58:46.397684 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f97cbd7-6fa4-4621-afa9-2347536c84fc None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:46.398220 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f97cbd7-6fa4-4621-afa9-2347536c84fc None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:46.398220 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f97cbd7-6fa4-4621-afa9-2347536c84fc None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:46.399570 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 543/1085] 10.222.0.62 () {54 vars in 927 bytes} [Wed Aug 24 05:58:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:58:46.405285 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ccbdb2-635b-4a19-aac6-f7f0719631bf None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:46.405708 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ccbdb2-635b-4a19-aac6-f7f0719631bf None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:46.405839 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ccbdb2-635b-4a19-aac6-f7f0719631bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:46.423787 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-91ccbdb2-635b-4a19-aac6-f7f0719631bf None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:58:46.424977 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-91ccbdb2-635b-4a19-aac6-f7f0719631bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:58:46.466687 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:46.490000 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 543/1086] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Aug 24 05:58:46 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 05:58:46.751618 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088881e8-3135-4f9d-8522-38865c570b3b None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:46.751936 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088881e8-3135-4f9d-8522-38865c570b3b None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:46.752215 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088881e8-3135-4f9d-8522-38865c570b3b None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:46.753667 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 544/1087] 10.222.0.62 () {54 vars in 927 bytes} [Wed Aug 24 05:58:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:58:46.754036 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ccbdb2-635b-4a19-aac6-f7f0719631bf None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:46.758746 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5404611-b477-4760-8e84-2e464b072f2d None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:46.759026 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5404611-b477-4760-8e84-2e464b072f2d None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:46.759295 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5404611-b477-4760-8e84-2e464b072f2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:46.777209 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-f5404611-b477-4760-8e84-2e464b072f2d None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:58:46.778424 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-f5404611-b477-4760-8e84-2e464b072f2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:58:46.805013 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:46.844212 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 545/1088] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Aug 24 05:58:46 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 05:58:46.852095 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ccbdb2-635b-4a19-aac6-f7f0719631bf None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:46.885761 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4dc61af-568c-42e7-85d0-903e316fd93b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:46.886378 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dc61af-568c-42e7-85d0-903e316fd93b None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:46.886654 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dc61af-568c-42e7-85d0-903e316fd93b None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:46.886922 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dc61af-568c-42e7-85d0-903e316fd93b None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:46.896657 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4dc61af-568c-42e7-85d0-903e316fd93b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:58:46.901221 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application [None req-f4dc61af-568c-42e7-85d0-903e316fd93b None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 05:58:46.901221 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:58:46.901221 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:58:46.901221 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:58:46.901221 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:58:46.901221 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:58:46.901221 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:58:46.901221 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:58:46.901221 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:58:46.901221 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:58:46.901221 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:58:46.901221 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:58:46.901221 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:58:46.901221 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:58:46.901221 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:58:46.901221 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:58:46.902189 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:58:46.902189 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:58:46.902189 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:58:46.902189 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:58:46.902189 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:58:46.902189 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:58:46.902189 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:58:46.902189 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:58:46.902189 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:58:46.902189 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:58:46.902189 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:58:46.902189 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application with Lock( Aug 24 05:58:46.902189 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:58:46.902189 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:58:46.902189 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:58:46.902189 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:58:46.903088 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:58:46.903088 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:58:46.903088 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:58:46.903088 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:58:46.903088 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:58:46.903088 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:58:46.903088 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:58:46.903088 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:58:46.903088 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:58:46.903088 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:58:46.903088 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 05:58:46.903088 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Aug 24 05:58:46.903088 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 544/1089] 10.222.0.62 () {56 vars in 1120 bytes} [Wed Aug 24 05:58:46 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 05:58:46.908306 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5404611-b477-4760-8e84-2e464b072f2d None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:47.003626 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5404611-b477-4760-8e84-2e464b072f2d None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:47.035986 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907540ca-9428-4287-99d9-b37ebb13f238 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:47.036620 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907540ca-9428-4287-99d9-b37ebb13f238 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:47.036900 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907540ca-9428-4287-99d9-b37ebb13f238 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:47.037169 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907540ca-9428-4287-99d9-b37ebb13f238 None admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:47.037823 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907540ca-9428-4287-99d9-b37ebb13f238 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:58:47.038124 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907540ca-9428-4287-99d9-b37ebb13f238 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:47.039487 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907540ca-9428-4287-99d9-b37ebb13f238 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:47.047924 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 546/1090] 10.222.0.62 () {56 vars in 1115 bytes} [Wed Aug 24 05:58:46 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:58:47.053559 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4dc61af-568c-42e7-85d0-903e316fd93b None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:47.087904 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:47.108442 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907540ca-9428-4287-99d9-b37ebb13f238 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:47.138198 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:47.154680 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4dc61af-568c-42e7-85d0-903e316fd93b None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:47.188883 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ed8011-955e-458b-974c-e6d3ac9ec29c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:47.189509 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ed8011-955e-458b-974c-e6d3ac9ec29c None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:47.189784 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ed8011-955e-458b-974c-e6d3ac9ec29c None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:47.190081 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ed8011-955e-458b-974c-e6d3ac9ec29c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:47.199659 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ed8011-955e-458b-974c-e6d3ac9ec29c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:47.201137 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ed8011-955e-458b-974c-e6d3ac9ec29c None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:47.212469 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907540ca-9428-4287-99d9-b37ebb13f238 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:47.215164 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application [None req-26ed8011-955e-458b-974c-e6d3ac9ec29c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:58:47.215164 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:58:47.215164 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:58:47.215164 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:58:47.215164 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:58:47.215164 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:58:47.215164 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:58:47.215164 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:58:47.215164 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:58:47.215164 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:58:47.215164 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:58:47.215164 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:58:47.215164 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 05:58:47.215164 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 05:58:47.215164 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 05:58:47.215164 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 05:58:47.216705 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:58:47.216705 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:58:47.216705 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:58:47.216705 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:58:47.216705 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:58:47.216705 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:58:47.216705 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:58:47.216705 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application with Lock( Aug 24 05:58:47.216705 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:58:47.216705 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:58:47.216705 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:58:47.216705 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:58:47.216705 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:58:47.216705 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:58:47.216705 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:58:47.216705 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:58:47.216705 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 05:58:47.218169 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 05:58:47.218169 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:58:47.218169 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Aug 24 05:58:47.218169 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 545/1091] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Aug 24 05:58:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:58:47.222913 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ed8011-955e-458b-974c-e6d3ac9ec29c None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:47.246990 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51acc1a-eefe-450f-ad39-e05062ce13f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:47.247609 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51acc1a-eefe-450f-ad39-e05062ce13f0 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:47.247884 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51acc1a-eefe-450f-ad39-e05062ce13f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:47.248148 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51acc1a-eefe-450f-ad39-e05062ce13f0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:47.258022 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51acc1a-eefe-450f-ad39-e05062ce13f0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:58:47.262614 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-c51acc1a-eefe-450f-ad39-e05062ce13f0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 05:58:47.262614 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:58:47.262614 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:58:47.262614 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:58:47.262614 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:58:47.262614 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:58:47.262614 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:58:47.262614 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:58:47.262614 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:58:47.262614 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:58:47.262614 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:58:47.262614 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:58:47.262614 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:58:47.262614 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:58:47.262614 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:58:47.262614 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:58:47.263715 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:58:47.263715 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:58:47.263715 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:58:47.263715 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:58:47.263715 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:58:47.263715 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:58:47.263715 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:58:47.263715 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:58:47.263715 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:58:47.263715 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:58:47.263715 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:58:47.263715 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 24 05:58:47.263715 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:58:47.263715 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:58:47.263715 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:58:47.263715 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:58:47.264598 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:58:47.264598 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:58:47.264598 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:58:47.264598 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:58:47.264598 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:58:47.264598 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:58:47.264598 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:58:47.264598 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:58:47.264598 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:58:47.264598 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:58:47.264598 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 05:58:47.264598 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 24 05:58:47.264598 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 547/1092] 10.222.0.62 () {56 vars in 1096 bytes} [Wed Aug 24 05:58:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 05:58:47.269583 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51acc1a-eefe-450f-ad39-e05062ce13f0 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:47.308150 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:47.319509 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ed8011-955e-458b-974c-e6d3ac9ec29c None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:47.352724 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:47.353990 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98970148-9c0d-43ae-a1d6-33db9f960e92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:47.354588 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98970148-9c0d-43ae-a1d6-33db9f960e92 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:47.354866 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98970148-9c0d-43ae-a1d6-33db9f960e92 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:47.355139 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98970148-9c0d-43ae-a1d6-33db9f960e92 None admin] PATH_INFO: `/v3/domains` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:47.355781 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98970148-9c0d-43ae-a1d6-33db9f960e92 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:58:47.356665 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98970148-9c0d-43ae-a1d6-33db9f960e92 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:47.358031 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98970148-9c0d-43ae-a1d6-33db9f960e92 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:47.368504 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51acc1a-eefe-450f-ad39-e05062ce13f0 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:47.377278 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 546/1093] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Aug 24 05:58:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:58:47.377278 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98970148-9c0d-43ae-a1d6-33db9f960e92 None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:47.402058 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3911947d-ca6e-473b-8724-1a023dbb87d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:47.402658 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3911947d-ca6e-473b-8724-1a023dbb87d9 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:47.403443 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3911947d-ca6e-473b-8724-1a023dbb87d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:47.403710 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3911947d-ca6e-473b-8724-1a023dbb87d9 None admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:47.404353 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3911947d-ca6e-473b-8724-1a023dbb87d9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:58:47.404673 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3911947d-ca6e-473b-8724-1a023dbb87d9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:47.406022 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3911947d-ca6e-473b-8724-1a023dbb87d9 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:47.413920 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 548/1094] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Aug 24 05:58:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:58:47.420400 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3911947d-ca6e-473b-8724-1a023dbb87d9 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:47.472769 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98970148-9c0d-43ae-a1d6-33db9f960e92 None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:47.505300 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498be335-8ed6-41d5-8b64-125b3a01fd25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:47.505925 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498be335-8ed6-41d5-8b64-125b3a01fd25 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:47.506201 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498be335-8ed6-41d5-8b64-125b3a01fd25 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:47.506472 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498be335-8ed6-41d5-8b64-125b3a01fd25 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:47.513832 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498be335-8ed6-41d5-8b64-125b3a01fd25 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:47.515196 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498be335-8ed6-41d5-8b64-125b3a01fd25 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:47.516809 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3911947d-ca6e-473b-8724-1a023dbb87d9 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:47.525916 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application [None req-498be335-8ed6-41d5-8b64-125b3a01fd25 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 05:58:47.525916 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:58:47.525916 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:58:47.525916 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:58:47.525916 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:58:47.525916 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:58:47.525916 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:58:47.525916 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:58:47.525916 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:58:47.525916 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:58:47.525916 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:58:47.525916 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:58:47.525916 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 05:58:47.525916 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 05:58:47.525916 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 05:58:47.525916 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 05:58:47.526918 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:58:47.526918 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:58:47.526918 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 05:58:47.526918 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 05:58:47.526918 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 05:58:47.526918 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 05:58:47.526918 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:58:47.526918 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:58:47.526918 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:58:47.526918 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:58:47.526918 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:58:47.526918 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application with Lock( Aug 24 05:58:47.526918 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:58:47.526918 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:58:47.526918 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:58:47.526918 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:58:47.526918 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:58:47.526918 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:58:47.529010 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:58:47.529010 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:58:47.529010 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 05:58:47.529010 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 05:58:47.529010 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 05:58:47.529010 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 05:58:47.529010 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 05:58:47.529010 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 05:58:47.529010 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 05:58:47.529010 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Aug 24 05:58:47.529010 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 547/1095] 10.222.0.62 () {56 vars in 1093 bytes} [Wed Aug 24 05:58:47 2022] GET /identity/v3/users/nova => generated 82 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 05:58:47.533144 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498be335-8ed6-41d5-8b64-125b3a01fd25 None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:47.545586 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:47.555598 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6a9ff9-aa50-4374-9e70-d0daa43f4a8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:47.560360 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6a9ff9-aa50-4374-9e70-d0daa43f4a8b None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:47.560360 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6a9ff9-aa50-4374-9e70-d0daa43f4a8b None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:47.560360 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6a9ff9-aa50-4374-9e70-d0daa43f4a8b None admin] PATH_INFO: `/v3/users/027c81b97b8a4e82a55477f1e5c555be` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:47.560360 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6a9ff9-aa50-4374-9e70-d0daa43f4a8b None admin] RBAC: Authorizing `identity:get_user(user_id=027c81b97b8a4e82a55477f1e5c555be)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:47.560360 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6a9ff9-aa50-4374-9e70-d0daa43f4a8b None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:47.560876 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 549/1096] 10.222.0.62 () {56 vars in 1177 bytes} [Wed Aug 24 05:58:47 2022] GET /identity/v3/users/027c81b97b8a4e82a55477f1e5c555be => generated 285 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:58:47.566554 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6a9ff9-aa50-4374-9e70-d0daa43f4a8b None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:47.584455 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:47.632657 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498be335-8ed6-41d5-8b64-125b3a01fd25 None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:47.664379 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6a9ff9-aa50-4374-9e70-d0daa43f4a8b None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:47.667742 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567bfc86-7190-47d6-b1fc-bf93930241de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:47.668410 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567bfc86-7190-47d6-b1fc-bf93930241de None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:47.668725 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567bfc86-7190-47d6-b1fc-bf93930241de None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:47.669002 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567bfc86-7190-47d6-b1fc-bf93930241de None admin] PATH_INFO: `/v3/users` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:47.669725 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567bfc86-7190-47d6-b1fc-bf93930241de None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:58:47.670316 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567bfc86-7190-47d6-b1fc-bf93930241de None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:47.671750 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567bfc86-7190-47d6-b1fc-bf93930241de None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:47.688588 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 548/1097] 10.222.0.62 () {56 vars in 1133 bytes} [Wed Aug 24 05:58:47 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:58:47.694302 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567bfc86-7190-47d6-b1fc-bf93930241de None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:47.703717 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b915a9-bde5-443d-8e5c-0f69a785fcbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:47.704323 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b915a9-bde5-443d-8e5c-0f69a785fcbf None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:47.704630 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b915a9-bde5-443d-8e5c-0f69a785fcbf None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:47.705093 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b915a9-bde5-443d-8e5c-0f69a785fcbf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:47.758418 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:47.775814 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b915a9-bde5-443d-8e5c-0f69a785fcbf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:47.775814 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b915a9-bde5-443d-8e5c-0f69a785fcbf None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:47.789795 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:47.793393 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567bfc86-7190-47d6-b1fc-bf93930241de None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:47.826947 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3814682-daa5-44e4-bc8e-15f281c305ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:47.827667 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3814682-daa5-44e4-bc8e-15f281c305ac None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:47.827951 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3814682-daa5-44e4-bc8e-15f281c305ac None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:47.828218 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3814682-daa5-44e4-bc8e-15f281c305ac None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:47.870729 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 550/1098] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Aug 24 05:58:47 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 24 05:58:47.876583 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b915a9-bde5-443d-8e5c-0f69a785fcbf None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:47.895048 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3814682-daa5-44e4-bc8e-15f281c305ac None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:47.896509 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3814682-daa5-44e4-bc8e-15f281c305ac None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:47.959363 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:47.973324 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b915a9-bde5-443d-8e5c-0f69a785fcbf None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:47.988670 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:47.993815 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 549/1099] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Aug 24 05:58:47 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 24 05:58:47.999686 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3814682-daa5-44e4-bc8e-15f281c305ac None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:48.009055 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e87438-474a-4982-8f6c-05d7fd487a38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:48.009683 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e87438-474a-4982-8f6c-05d7fd487a38 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:48.009962 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e87438-474a-4982-8f6c-05d7fd487a38 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:48.010234 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e87438-474a-4982-8f6c-05d7fd487a38 None admin] PATH_INFO: `/v3/projects/b68006cd169244c5b544c7e16d1e759d` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:48.011331 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e87438-474a-4982-8f6c-05d7fd487a38 None admin] RBAC: Authorizing `identity:get_project(project_id=b68006cd169244c5b544c7e16d1e759d)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:48.013416 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e87438-474a-4982-8f6c-05d7fd487a38 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:48.015037 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 551/1100] 10.222.0.62 () {56 vars in 1186 bytes} [Wed Aug 24 05:58:47 2022] GET /identity/v3/projects/b68006cd169244c5b544c7e16d1e759d => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:58:48.020917 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e87438-474a-4982-8f6c-05d7fd487a38 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:48.098594 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3814682-daa5-44e4-bc8e-15f281c305ac None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:48.117086 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e87438-474a-4982-8f6c-05d7fd487a38 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:48.132514 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1c9038-8f46-42c5-9dfe-95ffaeaabf2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:48.133156 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1c9038-8f46-42c5-9dfe-95ffaeaabf2c None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:48.133430 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1c9038-8f46-42c5-9dfe-95ffaeaabf2c None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:48.133700 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1c9038-8f46-42c5-9dfe-95ffaeaabf2c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:48.144176 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1c9038-8f46-42c5-9dfe-95ffaeaabf2c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:48.145637 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1c9038-8f46-42c5-9dfe-95ffaeaabf2c None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:48.150506 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c0d3eb-9f88-4ae9-ac70-e2dd5b4b52bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:48.151100 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c0d3eb-9f88-4ae9-ac70-e2dd5b4b52bd None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:48.151379 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c0d3eb-9f88-4ae9-ac70-e2dd5b4b52bd None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:48.151652 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c0d3eb-9f88-4ae9-ac70-e2dd5b4b52bd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:48.152369 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c0d3eb-9f88-4ae9-ac70-e2dd5b4b52bd None admin] RBAC: Adding query filter params (role.id=49e47cd157d743f6905f1db07ec878a7, scope.project.id=b68006cd169244c5b544c7e16d1e759d, user.id=027c81b97b8a4e82a55477f1e5c555be) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:58:48.152742 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c0d3eb-9f88-4ae9-ac70-e2dd5b4b52bd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:48.154148 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c0d3eb-9f88-4ae9-ac70-e2dd5b4b52bd None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:48.159679 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application [None req-8e1c9038-8f46-42c5-9dfe-95ffaeaabf2c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:58:48.159679 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:58:48.159679 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:58:48.159679 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:58:48.159679 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:58:48.159679 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:58:48.159679 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:58:48.159679 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:58:48.159679 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:58:48.159679 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:58:48.159679 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:58:48.159679 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:58:48.159679 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 05:58:48.159679 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 05:58:48.159679 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 05:58:48.159679 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 05:58:48.160647 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:58:48.160647 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:58:48.160647 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:58:48.160647 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:58:48.160647 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:58:48.160647 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:58:48.160647 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:58:48.160647 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application with Lock( Aug 24 05:58:48.160647 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:58:48.160647 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:58:48.160647 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:58:48.160647 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:58:48.160647 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:58:48.160647 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:58:48.160647 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:58:48.160647 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:58:48.160647 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 05:58:48.161559 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 05:58:48.161559 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:58:48.161559 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Aug 24 05:58:48.161743 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 552/1101] 10.222.0.62 () {56 vars in 1374 bytes} [Wed Aug 24 05:58:48 2022] GET /identity/v3/role_assignments?user.id=027c81b97b8a4e82a55477f1e5c555be&scope.project.id=b68006cd169244c5b544c7e16d1e759d&role.id=49e47cd157d743f6905f1db07ec878a7 => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:58:48.161814 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 550/1102] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Aug 24 05:58:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 05:58:48.167722 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c0d3eb-9f88-4ae9-ac70-e2dd5b4b52bd None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:48.221660 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:48.274481 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c0d3eb-9f88-4ae9-ac70-e2dd5b4b52bd None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:48.314627 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85be4a38-49f9-46ca-a60d-bef7b153f52c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:48.315239 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85be4a38-49f9-46ca-a60d-bef7b153f52c None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:48.315519 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85be4a38-49f9-46ca-a60d-bef7b153f52c None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:48.315790 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85be4a38-49f9-46ca-a60d-bef7b153f52c None admin] PATH_INFO: `/v3/domains` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:48.316426 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85be4a38-49f9-46ca-a60d-bef7b153f52c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:58:48.316816 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85be4a38-49f9-46ca-a60d-bef7b153f52c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:48.318176 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85be4a38-49f9-46ca-a60d-bef7b153f52c None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:48.331000 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 553/1103] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Aug 24 05:58:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:58:48.336746 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1c9038-8f46-42c5-9dfe-95ffaeaabf2c None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:48.358172 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:48.437010 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1c9038-8f46-42c5-9dfe-95ffaeaabf2c None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:48.468885 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bacfef8-7a87-4e6e-b3bb-10a831642946 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:48.469503 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bacfef8-7a87-4e6e-b3bb-10a831642946 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:48.469777 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bacfef8-7a87-4e6e-b3bb-10a831642946 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:48.470045 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bacfef8-7a87-4e6e-b3bb-10a831642946 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:48.476436 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bacfef8-7a87-4e6e-b3bb-10a831642946 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:48.477878 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bacfef8-7a87-4e6e-b3bb-10a831642946 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:48.489192 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application [None req-3bacfef8-7a87-4e6e-b3bb-10a831642946 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 05:58:48.489192 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:58:48.489192 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:58:48.489192 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:58:48.489192 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:58:48.489192 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:58:48.489192 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:58:48.489192 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:58:48.489192 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:58:48.489192 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:58:48.489192 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:58:48.489192 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:58:48.489192 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 05:58:48.489192 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 05:58:48.489192 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 05:58:48.490223 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 05:58:48.490223 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:58:48.490223 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:58:48.490223 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:58:48.490223 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:58:48.490223 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:58:48.490223 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:58:48.490223 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:58:48.490223 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application with Lock( Aug 24 05:58:48.490223 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:58:48.490223 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:58:48.490223 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:58:48.490223 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:58:48.490223 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:58:48.490223 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:58:48.490223 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:58:48.490223 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:58:48.491487 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 05:58:48.491487 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 05:58:48.491487 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 05:58:48.491487 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 05:58:48.491487 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 05:58:48.491487 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 05:58:48.491487 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 05:58:48.491487 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Aug 24 05:58:48.491487 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 551/1104] 10.222.0.62 () {56 vars in 1111 bytes} [Wed Aug 24 05:58:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:58:48.496041 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85be4a38-49f9-46ca-a60d-bef7b153f52c None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:48.593580 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85be4a38-49f9-46ca-a60d-bef7b153f52c None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:48.602511 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:48.627831 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9a86a7-d6a5-4c21-a821-edb47a1534a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:48.628354 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9a86a7-d6a5-4c21-a821-edb47a1534a5 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:48.628668 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9a86a7-d6a5-4c21-a821-edb47a1534a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:48.628942 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9a86a7-d6a5-4c21-a821-edb47a1534a5 None admin] PATH_INFO: `/v3/projects` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:48.629627 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9a86a7-d6a5-4c21-a821-edb47a1534a5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:58:48.629929 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9a86a7-d6a5-4c21-a821-edb47a1534a5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:48.632665 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9a86a7-d6a5-4c21-a821-edb47a1534a5 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:48.643065 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 554/1105] 10.222.0.62 () {56 vars in 1148 bytes} [Wed Aug 24 05:58:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:58:48.649048 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bacfef8-7a87-4e6e-b3bb-10a831642946 None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:48.729254 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:48.752634 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bacfef8-7a87-4e6e-b3bb-10a831642946 None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:48.786959 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968a5353-f488-4c87-a63d-d04f637d1f46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:48.787557 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968a5353-f488-4c87-a63d-d04f637d1f46 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:48.787838 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968a5353-f488-4c87-a63d-d04f637d1f46 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:48.788109 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968a5353-f488-4c87-a63d-d04f637d1f46 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:48.788886 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968a5353-f488-4c87-a63d-d04f637d1f46 None admin] RBAC: Adding query filter params (role.id=2499f7e98f3e498ab329637f00bb2da4, scope.project.id=181f4762037e454489d9552cc603179c, user.id=fb6d3743340241f98cacb9e0d063298c) {{(pid=81851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:58:48.789194 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968a5353-f488-4c87-a63d-d04f637d1f46 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:48.790618 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968a5353-f488-4c87-a63d-d04f637d1f46 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:48.808595 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 552/1106] 10.222.0.62 () {56 vars in 1374 bytes} [Wed Aug 24 05:58:48 2022] GET /identity/v3/role_assignments?user.id=fb6d3743340241f98cacb9e0d063298c&scope.project.id=181f4762037e454489d9552cc603179c&role.id=2499f7e98f3e498ab329637f00bb2da4 => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:58:50.083045 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cc7581-87d5-441e-b9ab-14a5309c7c96 None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:50.083552 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cc7581-87d5-441e-b9ab-14a5309c7c96 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:50.083552 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cc7581-87d5-441e-b9ab-14a5309c7c96 None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:50.084877 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 555/1107] 10.222.0.62 () {54 vars in 927 bytes} [Wed Aug 24 05:58:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:58:50.090690 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d070556-cab3-4a07-8098-cb88849b5790 None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:50.091116 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d070556-cab3-4a07-8098-cb88849b5790 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:50.091242 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d070556-cab3-4a07-8098-cb88849b5790 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:50.108941 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-6d070556-cab3-4a07-8098-cb88849b5790 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:58:50.110100 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-6d070556-cab3-4a07-8098-cb88849b5790 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:58:50.173999 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 553/1108] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Aug 24 05:58:50 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 05:58:50.447146 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cc7581-87d5-441e-b9ab-14a5309c7c96 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:50.546276 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cc7581-87d5-441e-b9ab-14a5309c7c96 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:50.579099 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7857a984-37a3-4012-b46a-bece36a791e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:50.579711 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7857a984-37a3-4012-b46a-bece36a791e3 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:50.579986 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7857a984-37a3-4012-b46a-bece36a791e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:50.580254 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7857a984-37a3-4012-b46a-bece36a791e3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:50.581372 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7857a984-37a3-4012-b46a-bece36a791e3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:50.582807 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7857a984-37a3-4012-b46a-bece36a791e3 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:50.584342 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 556/1109] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Aug 24 05:58:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:58:50.589949 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d070556-cab3-4a07-8098-cb88849b5790 None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:50.657573 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:50.687331 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d070556-cab3-4a07-8098-cb88849b5790 None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:50.723574 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ea4542-1988-45b1-9c7c-fe18e9a260a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:50.726182 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ea4542-1988-45b1-9c7c-fe18e9a260a4 None admin] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:50.726182 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ea4542-1988-45b1-9c7c-fe18e9a260a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:50.726182 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ea4542-1988-45b1-9c7c-fe18e9a260a4 None admin] PATH_INFO: `/v3/users` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:50.726182 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ea4542-1988-45b1-9c7c-fe18e9a260a4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:50.726735 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ea4542-1988-45b1-9c7c-fe18e9a260a4 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:50.814718 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 554/1110] 10.222.0.62 () {60 vars in 1132 bytes} [Wed Aug 24 05:58:50 2022] POST /identity/v3/users => generated 286 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:58:52.717187 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36589331-c41d-42b2-b09a-3505489e883c None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:52.717584 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36589331-c41d-42b2-b09a-3505489e883c None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:52.717730 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36589331-c41d-42b2-b09a-3505489e883c None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:52.719070 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 557/1111] 10.222.0.62 () {54 vars in 927 bytes} [Wed Aug 24 05:58:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:58:52.724358 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cfaaf5-beae-4386-90d0-ab64aa09527b None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:52.724758 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cfaaf5-beae-4386-90d0-ab64aa09527b None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:52.724944 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cfaaf5-beae-4386-90d0-ab64aa09527b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:52.743108 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-e4cfaaf5-beae-4386-90d0-ab64aa09527b None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:58:52.744284 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-e4cfaaf5-beae-4386-90d0-ab64aa09527b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:58:52.809465 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 555/1112] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Aug 24 05:58:52 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 05:58:53.078219 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36589331-c41d-42b2-b09a-3505489e883c None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:53.113540 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:53.190207 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36589331-c41d-42b2-b09a-3505489e883c None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:53.222020 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6038bb0-730f-4ad4-91b0-1be36c7ba498 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:53.222627 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6038bb0-730f-4ad4-91b0-1be36c7ba498 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:53.222901 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6038bb0-730f-4ad4-91b0-1be36c7ba498 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:53.223169 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6038bb0-730f-4ad4-91b0-1be36c7ba498 None admin] PATH_INFO: `/v3/roles/ba3c2c3cb98c424f9ece19903a59c53c` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:53.224533 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6038bb0-730f-4ad4-91b0-1be36c7ba498 None admin] RBAC: Authorizing `identity:get_role(role_id=ba3c2c3cb98c424f9ece19903a59c53c)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:53.225887 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6038bb0-730f-4ad4-91b0-1be36c7ba498 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:53.227181 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 558/1113] 10.222.0.62 () {56 vars in 1177 bytes} [Wed Aug 24 05:58:53 2022] GET /identity/v3/roles/ba3c2c3cb98c424f9ece19903a59c53c => generated 223 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:58:53.232950 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cfaaf5-beae-4386-90d0-ab64aa09527b None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:53.287599 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:53.328157 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cfaaf5-beae-4386-90d0-ab64aa09527b None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:53.360668 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bc2cae-8772-4a0e-987c-4e6479889888 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:53.361217 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bc2cae-8772-4a0e-987c-4e6479889888 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:53.361489 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bc2cae-8772-4a0e-987c-4e6479889888 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:53.361758 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bc2cae-8772-4a0e-987c-4e6479889888 None admin] PATH_INFO: `/v3/users/58adce31ce304ac8bfb5cf4e3a6c52cc` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:53.374756 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18bc2cae-8772-4a0e-987c-4e6479889888 None admin] RBAC: Authorizing `identity:get_user(user_id=58adce31ce304ac8bfb5cf4e3a6c52cc)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:53.376189 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18bc2cae-8772-4a0e-987c-4e6479889888 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:53.377785 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 556/1114] 10.222.0.62 () {56 vars in 1177 bytes} [Wed Aug 24 05:58:53 2022] GET /identity/v3/users/58adce31ce304ac8bfb5cf4e3a6c52cc => generated 286 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:58:53.383475 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6038bb0-730f-4ad4-91b0-1be36c7ba498 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:53.477591 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6038bb0-730f-4ad4-91b0-1be36c7ba498 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:53.479792 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:53.509749 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a281c83-d9de-46e5-b202-3348abaa8bec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:53.510338 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a281c83-d9de-46e5-b202-3348abaa8bec None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:53.510611 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a281c83-d9de-46e5-b202-3348abaa8bec None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:53.510878 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a281c83-d9de-46e5-b202-3348abaa8bec None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:53.575409 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a281c83-d9de-46e5-b202-3348abaa8bec None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:53.576856 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a281c83-d9de-46e5-b202-3348abaa8bec None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:53.668758 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 559/1115] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Aug 24 05:58:53 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 286 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 05:58:53.674806 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bc2cae-8772-4a0e-987c-4e6479889888 None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:53.771395 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bc2cae-8772-4a0e-987c-4e6479889888 None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:53.787090 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:53.804751 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef996d3b-1a04-452d-a90a-f0105bf68320 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:53.805373 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef996d3b-1a04-452d-a90a-f0105bf68320 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:53.805650 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef996d3b-1a04-452d-a90a-f0105bf68320 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:53.805921 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef996d3b-1a04-452d-a90a-f0105bf68320 None admin] PATH_INFO: `/v3/projects/b68006cd169244c5b544c7e16d1e759d` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:53.807004 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef996d3b-1a04-452d-a90a-f0105bf68320 None admin] RBAC: Authorizing `identity:get_project(project_id=b68006cd169244c5b544c7e16d1e759d)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:53.808482 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef996d3b-1a04-452d-a90a-f0105bf68320 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:53.810745 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 557/1116] 10.222.0.62 () {56 vars in 1186 bytes} [Wed Aug 24 05:58:53 2022] GET /identity/v3/projects/b68006cd169244c5b544c7e16d1e759d => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:58:53.816889 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a281c83-d9de-46e5-b202-3348abaa8bec None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:53.819929 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:53.911913 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a281c83-d9de-46e5-b202-3348abaa8bec None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:53.944305 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2631bd28-ea12-4b22-a79b-d46574e95521 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:53.944861 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2631bd28-ea12-4b22-a79b-d46574e95521 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:53.945137 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2631bd28-ea12-4b22-a79b-d46574e95521 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:53.945403 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2631bd28-ea12-4b22-a79b-d46574e95521 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:53.946103 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2631bd28-ea12-4b22-a79b-d46574e95521 None admin] RBAC: Adding query filter params (role.id=ba3c2c3cb98c424f9ece19903a59c53c, scope.project.id=b68006cd169244c5b544c7e16d1e759d, user.id=58adce31ce304ac8bfb5cf4e3a6c52cc) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:58:53.946402 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2631bd28-ea12-4b22-a79b-d46574e95521 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:53.947807 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2631bd28-ea12-4b22-a79b-d46574e95521 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:53.952992 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 560/1117] 10.222.0.62 () {56 vars in 1374 bytes} [Wed Aug 24 05:58:53 2022] GET /identity/v3/role_assignments?user.id=58adce31ce304ac8bfb5cf4e3a6c52cc&scope.project.id=b68006cd169244c5b544c7e16d1e759d&role.id=ba3c2c3cb98c424f9ece19903a59c53c => generated 259 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:58:55.810993 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6907ac51-f810-41ad-91f3-36202a062c47 None None] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:55.811596 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6907ac51-f810-41ad-91f3-36202a062c47 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:55.811596 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6907ac51-f810-41ad-91f3-36202a062c47 None None] PATH_INFO: `/` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:55.812889 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 558/1118] 10.222.0.62 () {54 vars in 927 bytes} [Wed Aug 24 05:58:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:58:55.817906 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62527012-09c3-45cd-91b3-1b7acffe7860 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:55.818311 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62527012-09c3-45cd-91b3-1b7acffe7860 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:55.818450 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62527012-09c3-45cd-91b3-1b7acffe7860 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:55.836529 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-62527012-09c3-45cd-91b3-1b7acffe7860 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:58:55.837688 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-62527012-09c3-45cd-91b3-1b7acffe7860 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:58:55.895764 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:55.900913 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 561/1119] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Aug 24 05:58:55 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 05:58:56.165097 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6907ac51-f810-41ad-91f3-36202a062c47 None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:56.262512 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6907ac51-f810-41ad-91f3-36202a062c47 None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:56.295429 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d24939-2465-493e-a82e-cdf275ea79aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:56.296030 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d24939-2465-493e-a82e-cdf275ea79aa None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:56.296305 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d24939-2465-493e-a82e-cdf275ea79aa None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:56.296625 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d24939-2465-493e-a82e-cdf275ea79aa None admin] PATH_INFO: `/v3/roles/ba3c2c3cb98c424f9ece19903a59c53c` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:56.297977 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d24939-2465-493e-a82e-cdf275ea79aa None admin] RBAC: Authorizing `identity:get_role(role_id=ba3c2c3cb98c424f9ece19903a59c53c)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:56.299325 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d24939-2465-493e-a82e-cdf275ea79aa None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:56.300620 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 559/1120] 10.222.0.62 () {56 vars in 1177 bytes} [Wed Aug 24 05:58:56 2022] GET /identity/v3/roles/ba3c2c3cb98c424f9ece19903a59c53c => generated 223 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:58:56.306171 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62527012-09c3-45cd-91b3-1b7acffe7860 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:56.401517 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62527012-09c3-45cd-91b3-1b7acffe7860 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:56.434925 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e184fc2-4051-402e-8ebd-d9a175952ee7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:56.435481 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e184fc2-4051-402e-8ebd-d9a175952ee7 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:56.435753 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e184fc2-4051-402e-8ebd-d9a175952ee7 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:56.436017 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e184fc2-4051-402e-8ebd-d9a175952ee7 None admin] PATH_INFO: `/v3/users/58adce31ce304ac8bfb5cf4e3a6c52cc` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:56.437130 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e184fc2-4051-402e-8ebd-d9a175952ee7 None admin] RBAC: Authorizing `identity:get_user(user_id=58adce31ce304ac8bfb5cf4e3a6c52cc)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:56.438543 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e184fc2-4051-402e-8ebd-d9a175952ee7 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:56.440086 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 562/1121] 10.222.0.62 () {56 vars in 1177 bytes} [Wed Aug 24 05:58:56 2022] GET /identity/v3/users/58adce31ce304ac8bfb5cf4e3a6c52cc => generated 286 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:58:56.445548 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d24939-2465-493e-a82e-cdf275ea79aa None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:56.497283 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:56.542489 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d24939-2465-493e-a82e-cdf275ea79aa None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:56.575494 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc18c09-0dbf-4fde-a91c-cf7193e71e13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:56.576096 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc18c09-0dbf-4fde-a91c-cf7193e71e13 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:56.576370 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc18c09-0dbf-4fde-a91c-cf7193e71e13 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:56.576666 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc18c09-0dbf-4fde-a91c-cf7193e71e13 None admin] PATH_INFO: `/v3/projects/b68006cd169244c5b544c7e16d1e759d` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:56.577769 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc18c09-0dbf-4fde-a91c-cf7193e71e13 None admin] RBAC: Authorizing `identity:get_project(project_id=b68006cd169244c5b544c7e16d1e759d)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:56.579176 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc18c09-0dbf-4fde-a91c-cf7193e71e13 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:56.580784 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 560/1122] 10.222.0.62 () {56 vars in 1186 bytes} [Wed Aug 24 05:58:56 2022] GET /identity/v3/projects/b68006cd169244c5b544c7e16d1e759d => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:58:56.586783 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e184fc2-4051-402e-8ebd-d9a175952ee7 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:56.650103 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:56.682993 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e184fc2-4051-402e-8ebd-d9a175952ee7 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:56.717122 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1df0a9-74d5-47e6-ac01-5e67a5f8003e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:56.717776 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1df0a9-74d5-47e6-ac01-5e67a5f8003e None admin] REQUEST_METHOD: `PUT` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:56.718054 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1df0a9-74d5-47e6-ac01-5e67a5f8003e None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:56.718327 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1df0a9-74d5-47e6-ac01-5e67a5f8003e None admin] PATH_INFO: `/v3/projects/b68006cd169244c5b544c7e16d1e759d/users/58adce31ce304ac8bfb5cf4e3a6c52cc/roles/ba3c2c3cb98c424f9ece19903a59c53c` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:56.720428 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1df0a9-74d5-47e6-ac01-5e67a5f8003e None admin] RBAC: Authorizing `identity:create_grant(project_id=b68006cd169244c5b544c7e16d1e759d, user_id=58adce31ce304ac8bfb5cf4e3a6c52cc, role_id=ba3c2c3cb98c424f9ece19903a59c53c)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:56.721879 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1df0a9-74d5-47e6-ac01-5e67a5f8003e None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:56.836301 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 563/1123] 10.222.0.62 () {58 vars in 1439 bytes} [Wed Aug 24 05:58:56 2022] PUT /identity/v3/projects/b68006cd169244c5b544c7e16d1e759d/users/58adce31ce304ac8bfb5cf4e3a6c52cc/roles/ba3c2c3cb98c424f9ece19903a59c53c => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:58:58.679829 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49e38e3-d663-44bc-9d2b-fc87f1f22383 None None] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:58.680416 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49e38e3-d663-44bc-9d2b-fc87f1f22383 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:58.680416 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49e38e3-d663-44bc-9d2b-fc87f1f22383 None None] PATH_INFO: `/` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:58.681712 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 561/1124] 10.222.0.62 () {54 vars in 927 bytes} [Wed Aug 24 05:58:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:58:58.687391 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38235f5-da98-48ea-aa92-9ec9eacb86e5 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:58.687841 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38235f5-da98-48ea-aa92-9ec9eacb86e5 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:58.687942 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38235f5-da98-48ea-aa92-9ec9eacb86e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:58.708255 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-d38235f5-da98-48ea-aa92-9ec9eacb86e5 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:58:58.709480 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-d38235f5-da98-48ea-aa92-9ec9eacb86e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:58:58.772389 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 564/1125] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Aug 24 05:58:58 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 05:58:59.037387 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49e38e3-d663-44bc-9d2b-fc87f1f22383 None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:59.136912 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49e38e3-d663-44bc-9d2b-fc87f1f22383 None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:59.156177 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:59.169239 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf92e1fc-411d-46a6-afea-533f64e90966 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:59.170971 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf92e1fc-411d-46a6-afea-533f64e90966 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:59.170971 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf92e1fc-411d-46a6-afea-533f64e90966 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:59.170971 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf92e1fc-411d-46a6-afea-533f64e90966 None admin] PATH_INFO: `/v3/roles/ba3c2c3cb98c424f9ece19903a59c53c` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:59.172076 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf92e1fc-411d-46a6-afea-533f64e90966 None admin] RBAC: Authorizing `identity:get_role(role_id=ba3c2c3cb98c424f9ece19903a59c53c)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:59.173520 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf92e1fc-411d-46a6-afea-533f64e90966 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:59.174884 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 562/1126] 10.222.0.62 () {56 vars in 1177 bytes} [Wed Aug 24 05:58:59 2022] GET /identity/v3/roles/ba3c2c3cb98c424f9ece19903a59c53c => generated 223 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:58:59.180637 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38235f5-da98-48ea-aa92-9ec9eacb86e5 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:59.235029 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:59.277537 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38235f5-da98-48ea-aa92-9ec9eacb86e5 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:59.310769 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff4512c-1dd9-4bd8-b869-ffc88767ea23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:59.311337 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff4512c-1dd9-4bd8-b869-ffc88767ea23 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:59.311616 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff4512c-1dd9-4bd8-b869-ffc88767ea23 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:59.311912 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff4512c-1dd9-4bd8-b869-ffc88767ea23 None admin] PATH_INFO: `/v3/users/58adce31ce304ac8bfb5cf4e3a6c52cc` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:59.313045 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff4512c-1dd9-4bd8-b869-ffc88767ea23 None admin] RBAC: Authorizing `identity:get_user(user_id=58adce31ce304ac8bfb5cf4e3a6c52cc)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:59.314469 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff4512c-1dd9-4bd8-b869-ffc88767ea23 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:59.316044 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 565/1127] 10.222.0.62 () {56 vars in 1177 bytes} [Wed Aug 24 05:58:59 2022] GET /identity/v3/users/58adce31ce304ac8bfb5cf4e3a6c52cc => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:58:59.321725 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf92e1fc-411d-46a6-afea-533f64e90966 None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:59.417812 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf92e1fc-411d-46a6-afea-533f64e90966 None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:59.450445 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de88a680-ae53-4f48-95c0-d7f568ab0606 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:59.451026 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de88a680-ae53-4f48-95c0-d7f568ab0606 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:59.451301 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de88a680-ae53-4f48-95c0-d7f568ab0606 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:59.451574 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de88a680-ae53-4f48-95c0-d7f568ab0606 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:59.503299 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:59.515787 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de88a680-ae53-4f48-95c0-d7f568ab0606 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:59.517280 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de88a680-ae53-4f48-95c0-d7f568ab0606 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:59.612745 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 563/1128] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Aug 24 05:58:59 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 05:58:59.618554 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff4512c-1dd9-4bd8-b869-ffc88767ea23 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:59.718455 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff4512c-1dd9-4bd8-b869-ffc88767ea23 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:59.741259 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:59.752121 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40b5924-a9fa-4ca1-9f30-d0d5115ff341 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:59.754317 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40b5924-a9fa-4ca1-9f30-d0d5115ff341 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:59.754317 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40b5924-a9fa-4ca1-9f30-d0d5115ff341 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:59.754317 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40b5924-a9fa-4ca1-9f30-d0d5115ff341 None admin] PATH_INFO: `/v3/projects/b68006cd169244c5b544c7e16d1e759d` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:59.754574 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40b5924-a9fa-4ca1-9f30-d0d5115ff341 None admin] RBAC: Authorizing `identity:get_project(project_id=b68006cd169244c5b544c7e16d1e759d)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:59.755939 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40b5924-a9fa-4ca1-9f30-d0d5115ff341 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:59.757617 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 566/1129] 10.222.0.62 () {56 vars in 1186 bytes} [Wed Aug 24 05:58:59 2022] GET /identity/v3/projects/b68006cd169244c5b544c7e16d1e759d => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:58:59.763768 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de88a680-ae53-4f48-95c0-d7f568ab0606 None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:59.847748 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:59.859755 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de88a680-ae53-4f48-95c0-d7f568ab0606 None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:59.892630 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac1a61d-c8c3-449f-821e-e61310514dab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:59.893166 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac1a61d-c8c3-449f-821e-e61310514dab None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:59.893444 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac1a61d-c8c3-449f-821e-e61310514dab None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:59.893714 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac1a61d-c8c3-449f-821e-e61310514dab None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:59.894451 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac1a61d-c8c3-449f-821e-e61310514dab None admin] RBAC: Adding query filter params (role.id=ba3c2c3cb98c424f9ece19903a59c53c, scope.project.id=b68006cd169244c5b544c7e16d1e759d, user.id=58adce31ce304ac8bfb5cf4e3a6c52cc) {{(pid=81851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:58:59.894751 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac1a61d-c8c3-449f-821e-e61310514dab None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:59.896161 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac1a61d-c8c3-449f-821e-e61310514dab None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:59.901711 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 564/1130] 10.222.0.62 () {56 vars in 1374 bytes} [Wed Aug 24 05:58:59 2022] GET /identity/v3/role_assignments?user.id=58adce31ce304ac8bfb5cf4e3a6c52cc&scope.project.id=b68006cd169244c5b544c7e16d1e759d&role.id=ba3c2c3cb98c424f9ece19903a59c53c => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:59:01.798154 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745f52d8-3277-40c1-b893-d19cfc9b38f0 None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:59:01.798691 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745f52d8-3277-40c1-b893-d19cfc9b38f0 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:59:01.798691 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745f52d8-3277-40c1-b893-d19cfc9b38f0 None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:59:01.800025 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 567/1131] 10.222.0.62 () {54 vars in 927 bytes} [Wed Aug 24 05:59:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:59:01.805187 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061cbf5a-cc84-4310-bf0c-21c6667cae3b None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:59:01.805580 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061cbf5a-cc84-4310-bf0c-21c6667cae3b None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:59:01.805737 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061cbf5a-cc84-4310-bf0c-21c6667cae3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:59:01.824442 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-061cbf5a-cc84-4310-bf0c-21c6667cae3b None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:59:01.826017 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-061cbf5a-cc84-4310-bf0c-21c6667cae3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:59:01.891415 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 565/1132] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Aug 24 05:59:01 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 05:59:02.161225 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745f52d8-3277-40c1-b893-d19cfc9b38f0 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:59:02.258272 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745f52d8-3277-40c1-b893-d19cfc9b38f0 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:59:02.290548 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5a28e4-f188-4ae9-a040-44025058b4d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:59:02.291156 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5a28e4-f188-4ae9-a040-44025058b4d9 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:59:02.291435 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5a28e4-f188-4ae9-a040-44025058b4d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:59:02.291705 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5a28e4-f188-4ae9-a040-44025058b4d9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:59:02.292854 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5a28e4-f188-4ae9-a040-44025058b4d9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:59:02.294282 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5a28e4-f188-4ae9-a040-44025058b4d9 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:59:02.296109 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 568/1133] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Aug 24 05:59:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:59:02.301852 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061cbf5a-cc84-4310-bf0c-21c6667cae3b None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:59:02.378681 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:59:02.396035 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061cbf5a-cc84-4310-bf0c-21c6667cae3b None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:59:02.429654 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3db5f23-0fa3-47b1-91ba-b0f9b7224c43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:59:02.430232 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3db5f23-0fa3-47b1-91ba-b0f9b7224c43 None admin] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:59:02.430510 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3db5f23-0fa3-47b1-91ba-b0f9b7224c43 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:59:02.430781 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3db5f23-0fa3-47b1-91ba-b0f9b7224c43 None admin] PATH_INFO: `/v3/projects` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:59:02.431409 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3db5f23-0fa3-47b1-91ba-b0f9b7224c43 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:59:02.432854 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3db5f23-0fa3-47b1-91ba-b0f9b7224c43 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:59:02.505466 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 566/1134] 10.222.0.62 () {60 vars in 1141 bytes} [Wed Aug 24 05:59:02 2022] POST /identity/v3/projects => generated 311 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:59:04.424992 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0a54b7-ac79-4be2-a714-1aa5b59afd44 None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:59:04.425549 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0a54b7-ac79-4be2-a714-1aa5b59afd44 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:59:04.425549 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0a54b7-ac79-4be2-a714-1aa5b59afd44 None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:59:04.426888 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 569/1135] 10.222.0.62 () {54 vars in 927 bytes} [Wed Aug 24 05:59:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:59:04.431981 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99722b66-1a68-47e3-94b9-a6700f6e6519 None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:59:04.441736 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99722b66-1a68-47e3-94b9-a6700f6e6519 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:59:04.441736 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99722b66-1a68-47e3-94b9-a6700f6e6519 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:59:04.451083 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-99722b66-1a68-47e3-94b9-a6700f6e6519 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:59:04.452242 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-99722b66-1a68-47e3-94b9-a6700f6e6519 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:59:04.515363 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 567/1136] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Aug 24 05:59:04 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 05:59:04.785019 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0a54b7-ac79-4be2-a714-1aa5b59afd44 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:59:04.845188 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:59:04.890459 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0a54b7-ac79-4be2-a714-1aa5b59afd44 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:59:04.923364 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e94720-2f05-447d-8871-38f08763cc7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:59:04.923979 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e94720-2f05-447d-8871-38f08763cc7b None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:59:04.924250 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e94720-2f05-447d-8871-38f08763cc7b None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:59:04.924537 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e94720-2f05-447d-8871-38f08763cc7b None admin] PATH_INFO: `/v3/domains/default` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:59:04.925623 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e94720-2f05-447d-8871-38f08763cc7b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:59:04.927042 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e94720-2f05-447d-8871-38f08763cc7b None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:59:04.928609 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 570/1137] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Aug 24 05:59:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:59:04.934867 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99722b66-1a68-47e3-94b9-a6700f6e6519 None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:59:04.992555 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:59:05.028794 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99722b66-1a68-47e3-94b9-a6700f6e6519 None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:59:05.062085 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7b10a9-87e3-48d2-a912-e0fdb1ff6b9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:59:05.062669 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7b10a9-87e3-48d2-a912-e0fdb1ff6b9a None admin] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:59:05.062945 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7b10a9-87e3-48d2-a912-e0fdb1ff6b9a None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:59:05.063212 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7b10a9-87e3-48d2-a912-e0fdb1ff6b9a None admin] PATH_INFO: `/v3/users` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:59:05.063834 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7b10a9-87e3-48d2-a912-e0fdb1ff6b9a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:59:05.065264 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7b10a9-87e3-48d2-a912-e0fdb1ff6b9a None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:59:05.138863 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 568/1138] 10.222.0.62 () {60 vars in 1132 bytes} [Wed Aug 24 05:59:04 2022] POST /identity/v3/users => generated 286 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:59:07.067218 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2eb533-cf3c-48b4-9656-8b711b6c188d None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:59:07.067724 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2eb533-cf3c-48b4-9656-8b711b6c188d None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:59:07.067724 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2eb533-cf3c-48b4-9656-8b711b6c188d None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:59:07.069057 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 571/1139] 10.222.0.62 () {54 vars in 927 bytes} [Wed Aug 24 05:59:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:59:07.074803 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187d9335-8ce3-4bdb-a644-d7a323773105 None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:59:07.075187 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187d9335-8ce3-4bdb-a644-d7a323773105 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:59:07.075355 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187d9335-8ce3-4bdb-a644-d7a323773105 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:59:07.093531 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-187d9335-8ce3-4bdb-a644-d7a323773105 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:59:07.094713 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-187d9335-8ce3-4bdb-a644-d7a323773105 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:59:07.158844 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 569/1140] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Aug 24 05:59:07 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 05:59:07.429859 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2eb533-cf3c-48b4-9656-8b711b6c188d None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:59:07.527733 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2eb533-cf3c-48b4-9656-8b711b6c188d None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:59:07.553016 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:59:07.561042 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bbc9aa-f1ec-49fd-b885-c9ca93ce78ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:59:07.577691 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bbc9aa-f1ec-49fd-b885-c9ca93ce78ce None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:59:07.577691 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bbc9aa-f1ec-49fd-b885-c9ca93ce78ce None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:59:07.577691 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bbc9aa-f1ec-49fd-b885-c9ca93ce78ce None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:59:07.577691 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bbc9aa-f1ec-49fd-b885-c9ca93ce78ce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:59:07.577691 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-f4bbc9aa-f1ec-49fd-b885-c9ca93ce78ce None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 05:59:07.577691 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:59:07.577691 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:59:07.577691 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:59:07.577691 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:59:07.578432 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:59:07.578432 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:59:07.578432 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:59:07.578432 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:59:07.578432 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:59:07.578432 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:59:07.578432 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:59:07.578432 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:59:07.578432 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:59:07.578432 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:59:07.578432 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:59:07.578432 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:59:07.578432 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:59:07.578432 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:59:07.578432 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:59:07.578432 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:59:07.579381 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:59:07.579381 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:59:07.579381 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:59:07.579381 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:59:07.579381 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:59:07.579381 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:59:07.579381 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 24 05:59:07.579381 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:59:07.579381 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:59:07.579381 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:59:07.579381 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:59:07.579381 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:59:07.579381 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:59:07.579381 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:59:07.579381 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:59:07.579381 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:59:07.579381 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:59:07.580367 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:59:07.580367 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:59:07.580367 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:59:07.580367 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:59:07.580367 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 05:59:07.580367 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 24 05:59:07.580367 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 572/1141] 10.222.0.62 () {56 vars in 1096 bytes} [Wed Aug 24 05:59:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:59:07.584050 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187d9335-8ce3-4bdb-a644-d7a323773105 None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:59:07.631991 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:59:07.681222 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187d9335-8ce3-4bdb-a644-d7a323773105 None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:59:07.715620 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40e5660-93d8-4c78-8478-a258f327b273 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:59:07.716251 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40e5660-93d8-4c78-8478-a258f327b273 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:59:07.716548 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40e5660-93d8-4c78-8478-a258f327b273 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:59:07.716821 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40e5660-93d8-4c78-8478-a258f327b273 None admin] PATH_INFO: `/v3/roles` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:59:07.717472 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40e5660-93d8-4c78-8478-a258f327b273 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:59:07.717772 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40e5660-93d8-4c78-8478-a258f327b273 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:59:07.719118 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40e5660-93d8-4c78-8478-a258f327b273 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:59:07.726861 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 570/1142] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Aug 24 05:59:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:59:07.733356 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bbc9aa-f1ec-49fd-b885-c9ca93ce78ce None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:59:07.829784 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bbc9aa-f1ec-49fd-b885-c9ca93ce78ce None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:59:07.862609 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78247dc3-854e-4227-b999-3d37f06f7f64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:59:07.863221 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78247dc3-854e-4227-b999-3d37f06f7f64 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:59:07.863499 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78247dc3-854e-4227-b999-3d37f06f7f64 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:59:07.863772 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78247dc3-854e-4227-b999-3d37f06f7f64 None admin] PATH_INFO: `/v3/users/3a7a9d3d069c43009c7a64c952886373` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:59:07.876680 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78247dc3-854e-4227-b999-3d37f06f7f64 None admin] RBAC: Authorizing `identity:get_user(user_id=3a7a9d3d069c43009c7a64c952886373)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:59:07.878203 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78247dc3-854e-4227-b999-3d37f06f7f64 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:59:07.879779 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 573/1143] 10.222.0.62 () {56 vars in 1177 bytes} [Wed Aug 24 05:59:07 2022] GET /identity/v3/users/3a7a9d3d069c43009c7a64c952886373 => generated 286 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:59:07.885699 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40e5660-93d8-4c78-8478-a258f327b273 None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:59:07.986092 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40e5660-93d8-4c78-8478-a258f327b273 None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:59:07.998316 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:59:08.019921 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c039983b-28a3-4423-b62b-f3fad7e4c15c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:59:08.020597 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c039983b-28a3-4423-b62b-f3fad7e4c15c None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:59:08.021124 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c039983b-28a3-4423-b62b-f3fad7e4c15c None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:59:08.021404 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c039983b-28a3-4423-b62b-f3fad7e4c15c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:59:08.088093 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c039983b-28a3-4423-b62b-f3fad7e4c15c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:59:08.090215 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c039983b-28a3-4423-b62b-f3fad7e4c15c None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:59:08.184192 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 571/1144] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Aug 24 05:59:07 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 05:59:08.190110 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78247dc3-854e-4227-b999-3d37f06f7f64 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:59:08.235011 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:59:08.286287 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78247dc3-854e-4227-b999-3d37f06f7f64 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:59:08.318778 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e0dd52-1dc7-4304-ae36-9f9a56add9c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:59:08.319397 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e0dd52-1dc7-4304-ae36-9f9a56add9c3 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:59:08.319676 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e0dd52-1dc7-4304-ae36-9f9a56add9c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:59:08.319949 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e0dd52-1dc7-4304-ae36-9f9a56add9c3 None admin] PATH_INFO: `/v3/projects/966f10359a504e7b9dd4d4fdf857ad8f` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:59:08.321074 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e0dd52-1dc7-4304-ae36-9f9a56add9c3 None admin] RBAC: Authorizing `identity:get_project(project_id=966f10359a504e7b9dd4d4fdf857ad8f)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:59:08.322489 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e0dd52-1dc7-4304-ae36-9f9a56add9c3 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:59:08.324682 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 574/1145] 10.222.0.62 () {56 vars in 1186 bytes} [Wed Aug 24 05:59:08 2022] GET /identity/v3/projects/966f10359a504e7b9dd4d4fdf857ad8f => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:59:08.330638 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c039983b-28a3-4423-b62b-f3fad7e4c15c None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:59:08.344207 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:59:08.425766 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c039983b-28a3-4423-b62b-f3fad7e4c15c None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:59:08.458888 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58607e6-aabb-4111-8220-f1c6afaa7ff6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:59:08.459477 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58607e6-aabb-4111-8220-f1c6afaa7ff6 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:59:08.459755 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58607e6-aabb-4111-8220-f1c6afaa7ff6 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:59:08.460026 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58607e6-aabb-4111-8220-f1c6afaa7ff6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:59:08.460756 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58607e6-aabb-4111-8220-f1c6afaa7ff6 None admin] RBAC: Adding query filter params (role.id=49e47cd157d743f6905f1db07ec878a7, scope.project.id=966f10359a504e7b9dd4d4fdf857ad8f, user.id=3a7a9d3d069c43009c7a64c952886373) {{(pid=81851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:59:08.461062 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58607e6-aabb-4111-8220-f1c6afaa7ff6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:59:08.462482 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58607e6-aabb-4111-8220-f1c6afaa7ff6 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:59:08.467656 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 572/1146] 10.222.0.62 () {56 vars in 1374 bytes} [Wed Aug 24 05:59:08 2022] GET /identity/v3/role_assignments?user.id=3a7a9d3d069c43009c7a64c952886373&scope.project.id=966f10359a504e7b9dd4d4fdf857ad8f&role.id=49e47cd157d743f6905f1db07ec878a7 => generated 259 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:59:10.335289 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54706a9-9889-498f-a84c-9fa2ebf260e8 None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:59:10.335863 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54706a9-9889-498f-a84c-9fa2ebf260e8 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:59:10.335863 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54706a9-9889-498f-a84c-9fa2ebf260e8 None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:59:10.337219 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 575/1147] 10.222.0.62 () {54 vars in 927 bytes} [Wed Aug 24 05:59:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:59:10.342293 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96cda0d-e6cb-42a1-bdbc-aa3cdf7f2e78 None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:59:10.342676 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96cda0d-e6cb-42a1-bdbc-aa3cdf7f2e78 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:59:10.342843 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96cda0d-e6cb-42a1-bdbc-aa3cdf7f2e78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:59:10.361034 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-a96cda0d-e6cb-42a1-bdbc-aa3cdf7f2e78 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:59:10.362233 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-a96cda0d-e6cb-42a1-bdbc-aa3cdf7f2e78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:59:10.428062 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 573/1148] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Aug 24 05:59:10 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 05:59:10.693706 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54706a9-9889-498f-a84c-9fa2ebf260e8 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:59:10.791260 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54706a9-9889-498f-a84c-9fa2ebf260e8 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:59:10.805258 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:59:10.824334 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4182cc07-dab5-4cbb-9897-38874b4f080c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:59:10.824982 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4182cc07-dab5-4cbb-9897-38874b4f080c None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:59:10.826802 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4182cc07-dab5-4cbb-9897-38874b4f080c None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:59:10.826802 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4182cc07-dab5-4cbb-9897-38874b4f080c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:59:10.835101 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4182cc07-dab5-4cbb-9897-38874b4f080c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:59:10.840362 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-4182cc07-dab5-4cbb-9897-38874b4f080c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 05:59:10.840362 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:59:10.840362 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:59:10.840362 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:59:10.840362 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:59:10.840362 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:59:10.840362 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:59:10.840362 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:59:10.840362 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:59:10.840362 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:59:10.840362 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:59:10.840362 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:59:10.840362 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:59:10.840362 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:59:10.840362 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:59:10.840362 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:59:10.841361 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:59:10.841361 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:59:10.841361 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:59:10.841361 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:59:10.841361 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:59:10.841361 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:59:10.841361 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:59:10.841361 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:59:10.841361 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:59:10.841361 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:59:10.841361 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:59:10.841361 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 24 05:59:10.841361 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:59:10.841361 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:59:10.841361 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:59:10.841361 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:59:10.842200 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:59:10.842200 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:59:10.842200 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:59:10.842200 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:59:10.842200 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:59:10.842200 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:59:10.842200 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:59:10.842200 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:59:10.842200 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:59:10.842200 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:59:10.842200 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 05:59:10.842200 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 24 05:59:10.842200 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 576/1149] 10.222.0.62 () {56 vars in 1096 bytes} [Wed Aug 24 05:59:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:59:10.847599 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96cda0d-e6cb-42a1-bdbc-aa3cdf7f2e78 None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:59:10.855701 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:59:10.942327 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96cda0d-e6cb-42a1-bdbc-aa3cdf7f2e78 None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:59:10.976078 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3eb0598-8c33-4978-8aee-b369c7dbfcf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:59:10.976638 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3eb0598-8c33-4978-8aee-b369c7dbfcf9 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:59:10.976918 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3eb0598-8c33-4978-8aee-b369c7dbfcf9 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:59:10.977187 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3eb0598-8c33-4978-8aee-b369c7dbfcf9 None admin] PATH_INFO: `/v3/roles` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:59:10.977842 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3eb0598-8c33-4978-8aee-b369c7dbfcf9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:59:10.978145 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3eb0598-8c33-4978-8aee-b369c7dbfcf9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:59:10.979502 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3eb0598-8c33-4978-8aee-b369c7dbfcf9 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:59:10.987289 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 574/1150] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Aug 24 05:59:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:59:10.993306 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4182cc07-dab5-4cbb-9897-38874b4f080c None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:59:11.088770 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4182cc07-dab5-4cbb-9897-38874b4f080c None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:59:11.120615 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ea8a86-cf83-4a39-82f1-23f4b0589f02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:59:11.121216 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ea8a86-cf83-4a39-82f1-23f4b0589f02 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:59:11.121491 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ea8a86-cf83-4a39-82f1-23f4b0589f02 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:59:11.121760 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ea8a86-cf83-4a39-82f1-23f4b0589f02 None admin] PATH_INFO: `/v3/users/3a7a9d3d069c43009c7a64c952886373` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:59:11.122839 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ea8a86-cf83-4a39-82f1-23f4b0589f02 None admin] RBAC: Authorizing `identity:get_user(user_id=3a7a9d3d069c43009c7a64c952886373)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:59:11.124252 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ea8a86-cf83-4a39-82f1-23f4b0589f02 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:59:11.125800 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 577/1151] 10.222.0.62 () {56 vars in 1177 bytes} [Wed Aug 24 05:59:10 2022] GET /identity/v3/users/3a7a9d3d069c43009c7a64c952886373 => generated 286 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:59:11.131295 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3eb0598-8c33-4978-8aee-b369c7dbfcf9 None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:59:11.196392 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:59:11.228056 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3eb0598-8c33-4978-8aee-b369c7dbfcf9 None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:59:11.260349 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce540307-730d-434e-82ff-b749a083d8bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:59:11.261602 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce540307-730d-434e-82ff-b749a083d8bc None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:59:11.261880 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce540307-730d-434e-82ff-b749a083d8bc None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:59:11.262154 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce540307-730d-434e-82ff-b749a083d8bc None admin] PATH_INFO: `/v3/projects/966f10359a504e7b9dd4d4fdf857ad8f` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:59:11.263251 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce540307-730d-434e-82ff-b749a083d8bc None admin] RBAC: Authorizing `identity:get_project(project_id=966f10359a504e7b9dd4d4fdf857ad8f)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:59:11.264701 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce540307-730d-434e-82ff-b749a083d8bc None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:59:11.266290 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 575/1152] 10.222.0.62 () {56 vars in 1186 bytes} [Wed Aug 24 05:59:11 2022] GET /identity/v3/projects/966f10359a504e7b9dd4d4fdf857ad8f => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:59:11.272263 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ea8a86-cf83-4a39-82f1-23f4b0589f02 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:59:11.313651 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:59:11.367625 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ea8a86-cf83-4a39-82f1-23f4b0589f02 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:59:11.400688 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9970c20-d2c2-4086-b9c6-65d243081baf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:59:11.401330 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9970c20-d2c2-4086-b9c6-65d243081baf None admin] REQUEST_METHOD: `PUT` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:59:11.401631 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9970c20-d2c2-4086-b9c6-65d243081baf None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:59:11.401902 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9970c20-d2c2-4086-b9c6-65d243081baf None admin] PATH_INFO: `/v3/projects/966f10359a504e7b9dd4d4fdf857ad8f/users/3a7a9d3d069c43009c7a64c952886373/roles/49e47cd157d743f6905f1db07ec878a7` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:59:11.403757 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9970c20-d2c2-4086-b9c6-65d243081baf None admin] RBAC: Authorizing `identity:create_grant(project_id=966f10359a504e7b9dd4d4fdf857ad8f, user_id=3a7a9d3d069c43009c7a64c952886373, role_id=49e47cd157d743f6905f1db07ec878a7)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:59:11.405211 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9970c20-d2c2-4086-b9c6-65d243081baf None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:59:11.504459 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 578/1153] 10.222.0.62 () {58 vars in 1439 bytes} [Wed Aug 24 05:59:11 2022] PUT /identity/v3/projects/966f10359a504e7b9dd4d4fdf857ad8f/users/3a7a9d3d069c43009c7a64c952886373/roles/49e47cd157d743f6905f1db07ec878a7 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:59:13.346396 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3317de42-d792-4f9c-81ec-969c7025f7a8 None None] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:59:13.346920 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3317de42-d792-4f9c-81ec-969c7025f7a8 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:59:13.347031 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3317de42-d792-4f9c-81ec-969c7025f7a8 None None] PATH_INFO: `/` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:59:13.348290 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 576/1154] 10.222.0.62 () {54 vars in 927 bytes} [Wed Aug 24 05:59:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:59:13.353679 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bef3ad-e94c-45b3-896e-3cb5fb9f5ab4 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:59:13.354096 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bef3ad-e94c-45b3-896e-3cb5fb9f5ab4 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:59:13.354200 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bef3ad-e94c-45b3-896e-3cb5fb9f5ab4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:59:13.371495 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-63bef3ad-e94c-45b3-896e-3cb5fb9f5ab4 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:59:13.372688 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-63bef3ad-e94c-45b3-896e-3cb5fb9f5ab4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:59:13.436608 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 579/1155] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Aug 24 05:59:13 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 05:59:13.700609 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3317de42-d792-4f9c-81ec-969c7025f7a8 None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:59:13.798901 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3317de42-d792-4f9c-81ec-969c7025f7a8 None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:59:13.831714 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8c647c-6d6a-4193-bae2-828512dc4cc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:59:13.832329 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8c647c-6d6a-4193-bae2-828512dc4cc7 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:59:13.832635 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8c647c-6d6a-4193-bae2-828512dc4cc7 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:59:13.832913 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8c647c-6d6a-4193-bae2-828512dc4cc7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:59:13.835463 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:59:13.843234 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8c647c-6d6a-4193-bae2-828512dc4cc7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:59:13.847842 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application [None req-8d8c647c-6d6a-4193-bae2-828512dc4cc7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 05:59:13.847842 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:59:13.847842 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:59:13.847842 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:59:13.847842 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:59:13.847842 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:59:13.847842 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:59:13.847842 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:59:13.847842 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:59:13.847842 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:59:13.847842 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:59:13.847842 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:59:13.847842 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:59:13.847842 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:59:13.847842 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:59:13.847842 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:59:13.854298 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:59:13.854298 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:59:13.854298 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:59:13.854298 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:59:13.854298 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:59:13.854298 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:59:13.854298 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:59:13.854298 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:59:13.854298 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:59:13.854298 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:59:13.854298 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:59:13.854298 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application with Lock( Aug 24 05:59:13.854298 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:59:13.854298 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:59:13.854298 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:59:13.854298 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:59:13.855691 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:59:13.855691 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:59:13.855691 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:59:13.855691 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:59:13.855691 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:59:13.855691 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:59:13.855691 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:59:13.855691 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:59:13.855691 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:59:13.855691 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:59:13.855691 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 05:59:13.855691 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Aug 24 05:59:13.855691 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 577/1156] 10.222.0.62 () {56 vars in 1096 bytes} [Wed Aug 24 05:59:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:59:13.856616 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bef3ad-e94c-45b3-896e-3cb5fb9f5ab4 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:59:13.885655 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:59:13.950917 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bef3ad-e94c-45b3-896e-3cb5fb9f5ab4 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:59:13.983728 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36db575f-1c7b-414b-83bb-286849d4ca80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:59:13.984329 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36db575f-1c7b-414b-83bb-286849d4ca80 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:59:13.986833 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36db575f-1c7b-414b-83bb-286849d4ca80 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:59:13.986833 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36db575f-1c7b-414b-83bb-286849d4ca80 None admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:59:13.986833 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36db575f-1c7b-414b-83bb-286849d4ca80 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:59:13.986833 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36db575f-1c7b-414b-83bb-286849d4ca80 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:59:13.987231 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36db575f-1c7b-414b-83bb-286849d4ca80 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:59:13.994989 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 580/1157] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Aug 24 05:59:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:59:14.001266 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8c647c-6d6a-4193-bae2-828512dc4cc7 None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:59:14.097370 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8c647c-6d6a-4193-bae2-828512dc4cc7 None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:59:14.130142 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7de18f9-4ff0-4574-babf-15f5d4e46ad6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:59:14.130746 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7de18f9-4ff0-4574-babf-15f5d4e46ad6 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:59:14.131020 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7de18f9-4ff0-4574-babf-15f5d4e46ad6 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:59:14.131289 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7de18f9-4ff0-4574-babf-15f5d4e46ad6 None admin] PATH_INFO: `/v3/users/3a7a9d3d069c43009c7a64c952886373` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:59:14.132376 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7de18f9-4ff0-4574-babf-15f5d4e46ad6 None admin] RBAC: Authorizing `identity:get_user(user_id=3a7a9d3d069c43009c7a64c952886373)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:59:14.133997 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7de18f9-4ff0-4574-babf-15f5d4e46ad6 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:59:14.135535 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 578/1158] 10.222.0.62 () {56 vars in 1177 bytes} [Wed Aug 24 05:59:14 2022] GET /identity/v3/users/3a7a9d3d069c43009c7a64c952886373 => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:59:14.141517 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36db575f-1c7b-414b-83bb-286849d4ca80 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:59:14.230868 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:59:14.237061 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36db575f-1c7b-414b-83bb-286849d4ca80 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:59:14.270293 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34dd09f-e4f8-4228-bbd7-a843ea599961 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:59:14.270927 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34dd09f-e4f8-4228-bbd7-a843ea599961 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:59:14.271408 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34dd09f-e4f8-4228-bbd7-a843ea599961 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:59:14.272025 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34dd09f-e4f8-4228-bbd7-a843ea599961 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:59:14.337435 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34dd09f-e4f8-4228-bbd7-a843ea599961 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:59:14.338938 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34dd09f-e4f8-4228-bbd7-a843ea599961 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:59:14.426459 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:59:14.433422 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 581/1159] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Aug 24 05:59:14 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 05:59:14.439423 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7de18f9-4ff0-4574-babf-15f5d4e46ad6 None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:59:14.499839 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:59:14.535806 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7de18f9-4ff0-4574-babf-15f5d4e46ad6 None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:59:14.568932 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b04fb47-7670-4955-b991-0c8063b722dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:59:14.569548 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b04fb47-7670-4955-b991-0c8063b722dc None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:59:14.569825 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b04fb47-7670-4955-b991-0c8063b722dc None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:59:14.570095 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b04fb47-7670-4955-b991-0c8063b722dc None admin] PATH_INFO: `/v3/projects/966f10359a504e7b9dd4d4fdf857ad8f` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:59:14.571187 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b04fb47-7670-4955-b991-0c8063b722dc None admin] RBAC: Authorizing `identity:get_project(project_id=966f10359a504e7b9dd4d4fdf857ad8f)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:59:14.572649 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b04fb47-7670-4955-b991-0c8063b722dc None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:59:14.574770 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 579/1160] 10.222.0.62 () {56 vars in 1186 bytes} [Wed Aug 24 05:59:14 2022] GET /identity/v3/projects/966f10359a504e7b9dd4d4fdf857ad8f => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:59:14.580872 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34dd09f-e4f8-4228-bbd7-a843ea599961 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:59:14.675500 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34dd09f-e4f8-4228-bbd7-a843ea599961 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:59:14.708237 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35371fc-6726-4bc1-b17f-f1f8470cfb02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:59:14.708881 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35371fc-6726-4bc1-b17f-f1f8470cfb02 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:59:14.709196 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35371fc-6726-4bc1-b17f-f1f8470cfb02 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:59:14.709501 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35371fc-6726-4bc1-b17f-f1f8470cfb02 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:59:14.710261 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35371fc-6726-4bc1-b17f-f1f8470cfb02 None admin] RBAC: Adding query filter params (role.id=49e47cd157d743f6905f1db07ec878a7, scope.project.id=966f10359a504e7b9dd4d4fdf857ad8f, user.id=3a7a9d3d069c43009c7a64c952886373) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:59:14.710584 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35371fc-6726-4bc1-b17f-f1f8470cfb02 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:59:14.712035 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35371fc-6726-4bc1-b17f-f1f8470cfb02 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:59:14.718287 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 582/1161] 10.222.0.62 () {56 vars in 1374 bytes} [Wed Aug 24 05:59:14 2022] GET /identity/v3/role_assignments?user.id=3a7a9d3d069c43009c7a64c952886373&scope.project.id=966f10359a504e7b9dd4d4fdf857ad8f&role.id=49e47cd157d743f6905f1db07ec878a7 => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:59:16.602096 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd151506-33f8-4f81-95bc-e1a9af337673 None None] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:59:16.602663 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd151506-33f8-4f81-95bc-e1a9af337673 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:59:16.602663 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd151506-33f8-4f81-95bc-e1a9af337673 None None] PATH_INFO: `/` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:59:16.603950 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 580/1162] 10.222.0.62 () {54 vars in 927 bytes} [Wed Aug 24 05:59:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:59:16.609212 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a21606-48e4-4e65-b2ef-46d68a88ed0f None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:59:16.609584 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a21606-48e4-4e65-b2ef-46d68a88ed0f None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:59:16.609841 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a21606-48e4-4e65-b2ef-46d68a88ed0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:59:16.628184 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-92a21606-48e4-4e65-b2ef-46d68a88ed0f None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:59:16.629430 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-92a21606-48e4-4e65-b2ef-46d68a88ed0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:59:16.680342 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:59:16.694137 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 583/1163] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Aug 24 05:59:16 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 05:59:16.961194 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd151506-33f8-4f81-95bc-e1a9af337673 None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:59:17.060663 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd151506-33f8-4f81-95bc-e1a9af337673 None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:59:17.088570 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:59:17.093211 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30665745-fcbc-4288-a588-7c559cc9f941 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:59:17.093804 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30665745-fcbc-4288-a588-7c559cc9f941 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:59:17.094085 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30665745-fcbc-4288-a588-7c559cc9f941 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:59:17.094356 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30665745-fcbc-4288-a588-7c559cc9f941 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:59:17.159308 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30665745-fcbc-4288-a588-7c559cc9f941 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:59:17.160774 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30665745-fcbc-4288-a588-7c559cc9f941 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:59:17.254265 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 581/1164] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Aug 24 05:59:16 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 05:59:17.260080 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a21606-48e4-4e65-b2ef-46d68a88ed0f None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:59:17.354820 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a21606-48e4-4e65-b2ef-46d68a88ed0f None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:59:17.387844 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1284800-d190-4326-abdb-3033e0e0f261 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:59:17.388473 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1284800-d190-4326-abdb-3033e0e0f261 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:59:17.388823 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1284800-d190-4326-abdb-3033e0e0f261 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:59:17.389130 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1284800-d190-4326-abdb-3033e0e0f261 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:59:17.395527 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1284800-d190-4326-abdb-3033e0e0f261 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:59:17.397029 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1284800-d190-4326-abdb-3033e0e0f261 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:59:17.407939 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-b1284800-d190-4326-abdb-3033e0e0f261 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 24 05:59:17.407939 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:59:17.407939 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Aug 24 05:59:17.407939 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Aug 24 05:59:17.407939 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 05:59:17.407939 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 05:59:17.407939 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 05:59:17.407939 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 05:59:17.407939 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Aug 24 05:59:17.407939 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 24 05:59:17.407939 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 05:59:17.407939 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 24 05:59:17.407939 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:59:17.407939 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:59:17.407939 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:59:17.407939 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:59:17.407939 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:59:17.409223 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:59:17.409223 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:59:17.409223 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:59:17.409223 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:59:17.409223 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:59:17.409223 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:59:17.409223 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 05:59:17.409223 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 05:59:17.409223 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 05:59:17.409223 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 05:59:17.409223 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:59:17.409223 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:59:17.409223 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:59:17.409223 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:59:17.409223 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:59:17.409223 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:59:17.410339 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:59:17.410339 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 24 05:59:17.410339 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:59:17.410339 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:59:17.410339 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:59:17.410339 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:59:17.410339 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:59:17.410339 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:59:17.410339 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:59:17.410339 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:59:17.410339 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Aug 24 05:59:17.410339 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 05:59:17.410339 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 24 05:59:17.410339 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 24 05:59:17.410339 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 584/1165] 10.222.0.62 () {56 vars in 1117 bytes} [Wed Aug 24 05:59:17 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:59:17.414746 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30665745-fcbc-4288-a588-7c559cc9f941 None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:59:17.444600 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:59:17.509607 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30665745-fcbc-4288-a588-7c559cc9f941 None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:59:17.542739 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e21612-5c79-4bd9-ba64-c85c46d1c2c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:59:17.543330 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e21612-5c79-4bd9-ba64-c85c46d1c2c9 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:59:17.543611 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e21612-5c79-4bd9-ba64-c85c46d1c2c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:59:17.543885 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e21612-5c79-4bd9-ba64-c85c46d1c2c9 None admin] PATH_INFO: `/v3/domains` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:59:17.544842 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e21612-5c79-4bd9-ba64-c85c46d1c2c9 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=81851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:59:17.545159 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e21612-5c79-4bd9-ba64-c85c46d1c2c9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:59:17.546507 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e21612-5c79-4bd9-ba64-c85c46d1c2c9 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:59:17.553425 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 582/1166] 10.222.0.62 () {56 vars in 1115 bytes} [Wed Aug 24 05:59:17 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:59:19.416360 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b586d337-04c6-49b5-aeee-2a4fa54e40b7 None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:59:19.416925 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b586d337-04c6-49b5-aeee-2a4fa54e40b7 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:59:19.416925 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b586d337-04c6-49b5-aeee-2a4fa54e40b7 None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:59:19.418850 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 585/1167] 10.222.0.62 () {54 vars in 927 bytes} [Wed Aug 24 05:59:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:59:19.424436 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223604b6-007d-43d7-b0d1-55043a7ae544 None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:59:19.424873 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223604b6-007d-43d7-b0d1-55043a7ae544 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:59:19.425011 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223604b6-007d-43d7-b0d1-55043a7ae544 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:59:19.442459 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-223604b6-007d-43d7-b0d1-55043a7ae544 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:59:19.443635 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-223604b6-007d-43d7-b0d1-55043a7ae544 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:59:19.508310 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 583/1168] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Aug 24 05:59:19 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 05:59:19.772840 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b586d337-04c6-49b5-aeee-2a4fa54e40b7 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:59:19.821725 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:59:19.871100 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b586d337-04c6-49b5-aeee-2a4fa54e40b7 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:59:19.903400 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe3484a-1f75-49c9-9573-4ded2c9793bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:59:19.903986 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe3484a-1f75-49c9-9573-4ded2c9793bc None admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:59:19.904265 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe3484a-1f75-49c9-9573-4ded2c9793bc None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:59:19.904553 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe3484a-1f75-49c9-9573-4ded2c9793bc None admin] PATH_INFO: `/v3/domains` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:59:19.905207 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe3484a-1f75-49c9-9573-4ded2c9793bc None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:59:19.906552 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe3484a-1f75-49c9-9573-4ded2c9793bc None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:59:19.985179 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 586/1169] 10.222.0.62 () {60 vars in 1138 bytes} [Wed Aug 24 05:59:19 2022] POST /identity/v3/domains => generated 267 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 05:59:21.882714 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a7498d-aa0c-486e-bd44-bfbbb604ed8c None None] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:59:21.883236 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a7498d-aa0c-486e-bd44-bfbbb604ed8c None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:59:21.883236 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a7498d-aa0c-486e-bd44-bfbbb604ed8c None None] PATH_INFO: `/` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:59:21.884616 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 584/1170] 10.222.0.62 () {54 vars in 927 bytes} [Wed Aug 24 05:59:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:59:21.889911 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21063d5-9917-4df2-bc4f-b5a0f8d270e9 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:59:21.890305 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21063d5-9917-4df2-bc4f-b5a0f8d270e9 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:59:21.890457 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21063d5-9917-4df2-bc4f-b5a0f8d270e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:59:21.909875 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-d21063d5-9917-4df2-bc4f-b5a0f8d270e9 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:59:21.911039 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-d21063d5-9917-4df2-bc4f-b5a0f8d270e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:59:21.975228 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 587/1171] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Aug 24 05:59:21 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 05:59:22.248297 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a7498d-aa0c-486e-bd44-bfbbb604ed8c None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:59:22.275333 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:59:22.346885 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a7498d-aa0c-486e-bd44-bfbbb604ed8c None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:59:22.378991 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f70a5f8-d71f-4690-9da7-825b5995bcc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:59:22.379599 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f70a5f8-d71f-4690-9da7-825b5995bcc4 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:59:22.379875 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f70a5f8-d71f-4690-9da7-825b5995bcc4 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:59:22.380146 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f70a5f8-d71f-4690-9da7-825b5995bcc4 None admin] PATH_INFO: `/v3/domains/1cae7755033e4f3e90bcd0e1fcbd06e9` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:59:22.390542 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f70a5f8-d71f-4690-9da7-825b5995bcc4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=1cae7755033e4f3e90bcd0e1fcbd06e9)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:59:22.391967 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f70a5f8-d71f-4690-9da7-825b5995bcc4 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:59:22.393545 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 585/1172] 10.222.0.62 () {56 vars in 1183 bytes} [Wed Aug 24 05:59:22 2022] GET /identity/v3/domains/1cae7755033e4f3e90bcd0e1fcbd06e9 => generated 267 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:59:22.399196 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21063d5-9917-4df2-bc4f-b5a0f8d270e9 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:59:22.437836 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:59:22.493897 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21063d5-9917-4df2-bc4f-b5a0f8d270e9 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:59:22.527248 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f52d83-e949-439f-b7bd-864cacee5498 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:59:22.527775 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f52d83-e949-439f-b7bd-864cacee5498 None admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:59:22.528050 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f52d83-e949-439f-b7bd-864cacee5498 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:59:22.528316 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f52d83-e949-439f-b7bd-864cacee5498 None admin] PATH_INFO: `/v3/projects` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:59:22.528979 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f52d83-e949-439f-b7bd-864cacee5498 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:59:22.530395 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f52d83-e949-439f-b7bd-864cacee5498 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:59:22.642471 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 588/1173] 10.222.0.62 () {60 vars in 1141 bytes} [Wed Aug 24 05:59:22 2022] POST /identity/v3/projects => generated 361 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:59:24.538058 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd599049-659d-422e-867a-f9c642000f4d None None] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:59:24.538597 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd599049-659d-422e-867a-f9c642000f4d None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:59:24.538597 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd599049-659d-422e-867a-f9c642000f4d None None] PATH_INFO: `/` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:59:24.539935 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 586/1174] 10.222.0.62 () {54 vars in 927 bytes} [Wed Aug 24 05:59:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:59:24.545311 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8b01ca-460d-48bc-93d6-e98a4c0c7541 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:59:24.545749 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8b01ca-460d-48bc-93d6-e98a4c0c7541 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:59:24.545843 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8b01ca-460d-48bc-93d6-e98a4c0c7541 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:59:24.563307 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-6a8b01ca-460d-48bc-93d6-e98a4c0c7541 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:59:24.564481 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-6a8b01ca-460d-48bc-93d6-e98a4c0c7541 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:59:24.628651 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 589/1175] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Aug 24 05:59:24 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 05:59:24.906298 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd599049-659d-422e-867a-f9c642000f4d None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:59:24.990453 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:59:25.006174 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd599049-659d-422e-867a-f9c642000f4d None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:59:25.039474 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f9508a-f246-4473-bd41-91f2abeb4c5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:59:25.040086 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f9508a-f246-4473-bd41-91f2abeb4c5b None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:59:25.040362 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f9508a-f246-4473-bd41-91f2abeb4c5b None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:59:25.040664 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f9508a-f246-4473-bd41-91f2abeb4c5b None admin] PATH_INFO: `/v3/domains/1cae7755033e4f3e90bcd0e1fcbd06e9` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:59:25.042249 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f9508a-f246-4473-bd41-91f2abeb4c5b None admin] RBAC: Authorizing `identity:get_domain(domain_id=1cae7755033e4f3e90bcd0e1fcbd06e9)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:59:25.043687 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f9508a-f246-4473-bd41-91f2abeb4c5b None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:59:25.045510 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 587/1176] 10.222.0.62 () {56 vars in 1183 bytes} [Wed Aug 24 05:59:24 2022] GET /identity/v3/domains/1cae7755033e4f3e90bcd0e1fcbd06e9 => generated 267 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:59:25.051260 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8b01ca-460d-48bc-93d6-e98a4c0c7541 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:59:25.072049 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:59:25.147837 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8b01ca-460d-48bc-93d6-e98a4c0c7541 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:59:25.180192 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df37859b-d131-48f1-b888-e9590fd4f519 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:59:25.180830 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df37859b-d131-48f1-b888-e9590fd4f519 None admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:59:25.181108 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df37859b-d131-48f1-b888-e9590fd4f519 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:59:25.181379 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df37859b-d131-48f1-b888-e9590fd4f519 None admin] PATH_INFO: `/v3/users` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:59:25.182010 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df37859b-d131-48f1-b888-e9590fd4f519 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:59:25.183516 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df37859b-d131-48f1-b888-e9590fd4f519 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:59:25.304123 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 590/1177] 10.222.0.62 () {60 vars in 1132 bytes} [Wed Aug 24 05:59:25 2022] POST /identity/v3/users => generated 311 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:59:27.214521 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10de7ecf-e0ed-492c-9159-e43a2de7e9cc None None] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:59:27.215079 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10de7ecf-e0ed-492c-9159-e43a2de7e9cc None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:59:27.215079 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10de7ecf-e0ed-492c-9159-e43a2de7e9cc None None] PATH_INFO: `/` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:59:27.216402 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 588/1178] 10.222.0.62 () {54 vars in 927 bytes} [Wed Aug 24 05:59:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:59:27.221733 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5d79d0-2e17-47e7-aff1-fcc202eda9d7 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:59:27.222120 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5d79d0-2e17-47e7-aff1-fcc202eda9d7 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:59:27.222277 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5d79d0-2e17-47e7-aff1-fcc202eda9d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:59:27.239796 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-0f5d79d0-2e17-47e7-aff1-fcc202eda9d7 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:59:27.241004 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-0f5d79d0-2e17-47e7-aff1-fcc202eda9d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:59:27.306055 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 591/1179] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Aug 24 05:59:27 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 05:59:27.572200 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10de7ecf-e0ed-492c-9159-e43a2de7e9cc None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:59:27.670519 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10de7ecf-e0ed-492c-9159-e43a2de7e9cc None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:59:27.702755 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f526597-3fd5-44da-8f98-a84afb84ae92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:59:27.703371 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f526597-3fd5-44da-8f98-a84afb84ae92 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:59:27.703648 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f526597-3fd5-44da-8f98-a84afb84ae92 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:59:27.703919 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f526597-3fd5-44da-8f98-a84afb84ae92 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:59:27.714320 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f526597-3fd5-44da-8f98-a84afb84ae92 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:59:27.718951 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application [None req-4f526597-3fd5-44da-8f98-a84afb84ae92 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 05:59:27.718951 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:59:27.718951 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:59:27.718951 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:59:27.718951 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:59:27.718951 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:59:27.718951 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:59:27.718951 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:59:27.718951 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:59:27.718951 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:59:27.718951 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:59:27.718951 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:59:27.718951 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:59:27.718951 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:59:27.718951 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:59:27.718951 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:59:27.720109 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:59:27.720109 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:59:27.720109 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:59:27.720109 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:59:27.720109 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:59:27.720109 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:59:27.720109 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:59:27.720109 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:59:27.720109 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:59:27.720109 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:59:27.720109 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:59:27.720109 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application with Lock( Aug 24 05:59:27.720109 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:59:27.720109 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:59:27.720109 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:59:27.720109 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:59:27.721313 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:59:27.721313 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:59:27.721313 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:59:27.721313 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:59:27.721313 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:59:27.721313 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:59:27.721313 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:59:27.721313 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:59:27.721313 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:59:27.721313 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:59:27.721313 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 05:59:27.721313 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Aug 24 05:59:27.721313 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 589/1180] 10.222.0.62 () {56 vars in 1096 bytes} [Wed Aug 24 05:59:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:59:27.726024 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5d79d0-2e17-47e7-aff1-fcc202eda9d7 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:59:27.734691 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:59:27.823235 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5d79d0-2e17-47e7-aff1-fcc202eda9d7 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:59:27.857663 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0f8158-8faa-4892-84cf-3412968246bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:59:27.858270 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0f8158-8faa-4892-84cf-3412968246bc None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:59:27.858550 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0f8158-8faa-4892-84cf-3412968246bc None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:59:27.858824 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0f8158-8faa-4892-84cf-3412968246bc None admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:59:27.859482 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0f8158-8faa-4892-84cf-3412968246bc None admin] RBAC: Adding query filter params (name=admin) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:59:27.859786 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0f8158-8faa-4892-84cf-3412968246bc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:59:27.861199 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0f8158-8faa-4892-84cf-3412968246bc None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:59:27.869291 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 592/1181] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Aug 24 05:59:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:59:27.875840 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f526597-3fd5-44da-8f98-a84afb84ae92 None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:59:27.892413 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:59:27.974170 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f526597-3fd5-44da-8f98-a84afb84ae92 None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:59:28.006729 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc08577d-22d3-46a9-ab01-69a474cfcd0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:59:28.007338 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc08577d-22d3-46a9-ab01-69a474cfcd0e None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:59:28.007618 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc08577d-22d3-46a9-ab01-69a474cfcd0e None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:59:28.007892 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc08577d-22d3-46a9-ab01-69a474cfcd0e None admin] PATH_INFO: `/v3/users/41adc23fb59a46d3a235e4ef37effe7e` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:59:28.020932 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc08577d-22d3-46a9-ab01-69a474cfcd0e None admin] RBAC: Authorizing `identity:get_user(user_id=41adc23fb59a46d3a235e4ef37effe7e)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:59:28.022367 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc08577d-22d3-46a9-ab01-69a474cfcd0e None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:59:28.023926 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 590/1182] 10.222.0.62 () {56 vars in 1177 bytes} [Wed Aug 24 05:59:27 2022] GET /identity/v3/users/41adc23fb59a46d3a235e4ef37effe7e => generated 311 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:59:28.029746 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0f8158-8faa-4892-84cf-3412968246bc None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:59:28.093901 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:59:28.125768 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0f8158-8faa-4892-84cf-3412968246bc None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:59:28.157470 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2daaafe0-8e9e-4523-b1ce-bd81697e4f1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:59:28.158052 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2daaafe0-8e9e-4523-b1ce-bd81697e4f1d None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:59:28.158326 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2daaafe0-8e9e-4523-b1ce-bd81697e4f1d None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:59:28.158592 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2daaafe0-8e9e-4523-b1ce-bd81697e4f1d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:59:28.221964 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2daaafe0-8e9e-4523-b1ce-bd81697e4f1d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:59:28.223392 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2daaafe0-8e9e-4523-b1ce-bd81697e4f1d None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:59:28.284151 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:59:28.317312 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 593/1183] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Aug 24 05:59:28 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 05:59:28.322792 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc08577d-22d3-46a9-ab01-69a474cfcd0e None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:59:28.390075 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:59:28.419154 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc08577d-22d3-46a9-ab01-69a474cfcd0e None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:59:28.451691 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec36e81d-77fd-4902-a6dd-0b0a758aa662 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:59:28.452308 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec36e81d-77fd-4902-a6dd-0b0a758aa662 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:59:28.452638 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec36e81d-77fd-4902-a6dd-0b0a758aa662 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:59:28.452916 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec36e81d-77fd-4902-a6dd-0b0a758aa662 None admin] PATH_INFO: `/v3/projects/61dd43d6ff6b4bdabc5e3cb34b2407ef` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:59:28.454021 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec36e81d-77fd-4902-a6dd-0b0a758aa662 None admin] RBAC: Authorizing `identity:get_project(project_id=61dd43d6ff6b4bdabc5e3cb34b2407ef)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:59:28.455447 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec36e81d-77fd-4902-a6dd-0b0a758aa662 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:59:28.457079 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 591/1184] 10.222.0.62 () {56 vars in 1186 bytes} [Wed Aug 24 05:59:28 2022] GET /identity/v3/projects/61dd43d6ff6b4bdabc5e3cb34b2407ef => generated 361 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:59:28.463050 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2daaafe0-8e9e-4523-b1ce-bd81697e4f1d None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:59:28.559384 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2daaafe0-8e9e-4523-b1ce-bd81697e4f1d None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:59:28.591953 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea94aed-ecd4-42aa-b567-12b58b99c219 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:59:28.592566 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea94aed-ecd4-42aa-b567-12b58b99c219 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:59:28.592848 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea94aed-ecd4-42aa-b567-12b58b99c219 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:59:28.593122 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea94aed-ecd4-42aa-b567-12b58b99c219 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:59:28.593830 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea94aed-ecd4-42aa-b567-12b58b99c219 None admin] RBAC: Adding query filter params (role.id=49e47cd157d743f6905f1db07ec878a7, scope.project.id=61dd43d6ff6b4bdabc5e3cb34b2407ef, user.id=41adc23fb59a46d3a235e4ef37effe7e) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:59:28.594138 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea94aed-ecd4-42aa-b567-12b58b99c219 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:59:28.595554 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea94aed-ecd4-42aa-b567-12b58b99c219 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:59:28.600721 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 594/1185] 10.222.0.62 () {56 vars in 1374 bytes} [Wed Aug 24 05:59:28 2022] GET /identity/v3/role_assignments?user.id=41adc23fb59a46d3a235e4ef37effe7e&scope.project.id=61dd43d6ff6b4bdabc5e3cb34b2407ef&role.id=49e47cd157d743f6905f1db07ec878a7 => generated 259 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:59:30.437544 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f1df98-85f5-44e8-8bd2-989e4dc8c179 None None] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:59:30.438116 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f1df98-85f5-44e8-8bd2-989e4dc8c179 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:59:30.438116 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f1df98-85f5-44e8-8bd2-989e4dc8c179 None None] PATH_INFO: `/` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:59:30.439398 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 592/1186] 10.222.0.62 () {54 vars in 927 bytes} [Wed Aug 24 05:59:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:59:30.444445 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3801d4e8-4466-4d65-82b9-3b9146c269f3 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:59:30.444869 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3801d4e8-4466-4d65-82b9-3b9146c269f3 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:59:30.445056 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3801d4e8-4466-4d65-82b9-3b9146c269f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:59:30.463020 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-3801d4e8-4466-4d65-82b9-3b9146c269f3 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:59:30.464857 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-3801d4e8-4466-4d65-82b9-3b9146c269f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:59:30.489429 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:59:30.528679 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 595/1187] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Aug 24 05:59:30 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 05:59:30.792767 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f1df98-85f5-44e8-8bd2-989e4dc8c179 None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:59:30.893837 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f1df98-85f5-44e8-8bd2-989e4dc8c179 None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:59:30.927056 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4133ea-5355-494d-9369-2b048a88414d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:59:30.927666 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4133ea-5355-494d-9369-2b048a88414d None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:59:30.927969 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4133ea-5355-494d-9369-2b048a88414d None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:59:30.928241 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4133ea-5355-494d-9369-2b048a88414d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:59:30.936199 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:59:30.939218 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4133ea-5355-494d-9369-2b048a88414d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:59:30.943888 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application [None req-9a4133ea-5355-494d-9369-2b048a88414d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 05:59:30.943888 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:59:30.943888 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:59:30.943888 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:59:30.943888 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:59:30.943888 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:59:30.943888 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:59:30.943888 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:59:30.943888 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:59:30.943888 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:59:30.943888 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:59:30.943888 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:59:30.943888 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:59:30.943888 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:59:30.943888 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:59:30.943888 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:59:30.956153 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:59:30.956153 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:59:30.956153 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:59:30.956153 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:59:30.956153 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:59:30.956153 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:59:30.956153 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:59:30.956153 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:59:30.956153 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:59:30.956153 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:59:30.956153 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:59:30.956153 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application with Lock( Aug 24 05:59:30.956153 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:59:30.956153 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:59:30.956153 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:59:30.956153 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:59:30.957096 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:59:30.957096 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:59:30.957096 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:59:30.957096 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:59:30.957096 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:59:30.957096 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:59:30.957096 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:59:30.957096 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:59:30.957096 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:59:30.957096 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:59:30.957096 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 05:59:30.957096 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Aug 24 05:59:30.957096 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 593/1188] 10.222.0.62 () {56 vars in 1096 bytes} [Wed Aug 24 05:59:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:59:30.957824 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3801d4e8-4466-4d65-82b9-3b9146c269f3 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:59:31.047059 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3801d4e8-4466-4d65-82b9-3b9146c269f3 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:59:31.079613 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e866fa-f0fd-4393-b6de-7ba0f56e9ecd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:59:31.080207 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e866fa-f0fd-4393-b6de-7ba0f56e9ecd None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:59:31.080484 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e866fa-f0fd-4393-b6de-7ba0f56e9ecd None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:59:31.080779 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e866fa-f0fd-4393-b6de-7ba0f56e9ecd None admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:59:31.081430 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e866fa-f0fd-4393-b6de-7ba0f56e9ecd None admin] RBAC: Adding query filter params (name=admin) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:59:31.081729 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e866fa-f0fd-4393-b6de-7ba0f56e9ecd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:59:31.083082 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e866fa-f0fd-4393-b6de-7ba0f56e9ecd None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:59:31.090818 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 596/1189] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Aug 24 05:59:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:59:31.096581 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4133ea-5355-494d-9369-2b048a88414d None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:59:31.193874 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4133ea-5355-494d-9369-2b048a88414d None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:59:31.225821 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1be2c59-655f-4ac5-9791-5584f9a10977 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:59:31.226367 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1be2c59-655f-4ac5-9791-5584f9a10977 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:59:31.226643 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1be2c59-655f-4ac5-9791-5584f9a10977 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:59:31.226912 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1be2c59-655f-4ac5-9791-5584f9a10977 None admin] PATH_INFO: `/v3/users/41adc23fb59a46d3a235e4ef37effe7e` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:59:31.228009 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1be2c59-655f-4ac5-9791-5584f9a10977 None admin] RBAC: Authorizing `identity:get_user(user_id=41adc23fb59a46d3a235e4ef37effe7e)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:59:31.230015 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1be2c59-655f-4ac5-9791-5584f9a10977 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:59:31.231530 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 594/1190] 10.222.0.62 () {56 vars in 1177 bytes} [Wed Aug 24 05:59:31 2022] GET /identity/v3/users/41adc23fb59a46d3a235e4ef37effe7e => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:59:31.237044 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e866fa-f0fd-4393-b6de-7ba0f56e9ecd None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:59:31.262405 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:59:31.334957 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e866fa-f0fd-4393-b6de-7ba0f56e9ecd None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:59:31.367553 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638d045d-1b5d-4389-8794-0b3749a2e496 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:59:31.368170 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638d045d-1b5d-4389-8794-0b3749a2e496 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:59:31.368446 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638d045d-1b5d-4389-8794-0b3749a2e496 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:59:31.368750 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638d045d-1b5d-4389-8794-0b3749a2e496 None admin] PATH_INFO: `/v3/projects/61dd43d6ff6b4bdabc5e3cb34b2407ef` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:59:31.370483 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638d045d-1b5d-4389-8794-0b3749a2e496 None admin] RBAC: Authorizing `identity:get_project(project_id=61dd43d6ff6b4bdabc5e3cb34b2407ef)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:59:31.371981 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638d045d-1b5d-4389-8794-0b3749a2e496 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:59:31.373646 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 597/1191] 10.222.0.62 () {56 vars in 1186 bytes} [Wed Aug 24 05:59:31 2022] GET /identity/v3/projects/61dd43d6ff6b4bdabc5e3cb34b2407ef => generated 361 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:59:31.379563 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1be2c59-655f-4ac5-9791-5584f9a10977 None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:59:31.444865 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:59:31.477577 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1be2c59-655f-4ac5-9791-5584f9a10977 None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:59:31.510217 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c66605-9db5-466b-8b07-9cc0da6e4bec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:59:31.510864 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c66605-9db5-466b-8b07-9cc0da6e4bec None admin] REQUEST_METHOD: `PUT` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:59:31.511145 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c66605-9db5-466b-8b07-9cc0da6e4bec None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:59:31.511421 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c66605-9db5-466b-8b07-9cc0da6e4bec None admin] PATH_INFO: `/v3/projects/61dd43d6ff6b4bdabc5e3cb34b2407ef/users/41adc23fb59a46d3a235e4ef37effe7e/roles/49e47cd157d743f6905f1db07ec878a7` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:59:31.513321 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c66605-9db5-466b-8b07-9cc0da6e4bec None admin] RBAC: Authorizing `identity:create_grant(project_id=61dd43d6ff6b4bdabc5e3cb34b2407ef, user_id=41adc23fb59a46d3a235e4ef37effe7e, role_id=49e47cd157d743f6905f1db07ec878a7)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:59:31.514746 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c66605-9db5-466b-8b07-9cc0da6e4bec None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:59:31.607579 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 595/1192] 10.222.0.62 () {58 vars in 1439 bytes} [Wed Aug 24 05:59:31 2022] PUT /identity/v3/projects/61dd43d6ff6b4bdabc5e3cb34b2407ef/users/41adc23fb59a46d3a235e4ef37effe7e/roles/49e47cd157d743f6905f1db07ec878a7 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:59:33.471716 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a600e5-5a66-4ab2-9d51-8793abea5a90 None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:59:33.472251 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a600e5-5a66-4ab2-9d51-8793abea5a90 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:59:33.472251 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a600e5-5a66-4ab2-9d51-8793abea5a90 None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:59:33.473627 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 598/1193] 10.222.0.62 () {54 vars in 927 bytes} [Wed Aug 24 05:59:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:59:33.479085 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc877739-b0f2-4575-9b4f-59d23f8b0910 None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:59:33.479500 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc877739-b0f2-4575-9b4f-59d23f8b0910 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:59:33.479640 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc877739-b0f2-4575-9b4f-59d23f8b0910 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:59:33.497270 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-dc877739-b0f2-4575-9b4f-59d23f8b0910 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:59:33.498452 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-dc877739-b0f2-4575-9b4f-59d23f8b0910 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:59:33.562565 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 596/1194] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Aug 24 05:59:33 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 05:59:33.827267 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a600e5-5a66-4ab2-9d51-8793abea5a90 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:59:33.916660 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:59:33.926188 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a600e5-5a66-4ab2-9d51-8793abea5a90 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:59:33.959712 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455b7a5d-6d3c-409a-8023-c9c9077bdf2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:59:33.960327 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455b7a5d-6d3c-409a-8023-c9c9077bdf2e None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:59:33.960634 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455b7a5d-6d3c-409a-8023-c9c9077bdf2e None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:59:33.960909 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455b7a5d-6d3c-409a-8023-c9c9077bdf2e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:59:33.970672 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455b7a5d-6d3c-409a-8023-c9c9077bdf2e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:59:33.975307 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-455b7a5d-6d3c-409a-8023-c9c9077bdf2e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 05:59:33.975307 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:59:33.975307 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:59:33.975307 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:59:33.975307 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:59:33.975307 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:59:33.975307 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:59:33.975307 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:59:33.975307 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:59:33.975307 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:59:33.975307 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:59:33.975307 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:59:33.975307 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:59:33.975307 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:59:33.975307 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:59:33.975307 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:59:33.976368 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:59:33.976368 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:59:33.976368 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:59:33.976368 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:59:33.976368 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:59:33.976368 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:59:33.976368 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:59:33.976368 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:59:33.976368 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:59:33.976368 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:59:33.976368 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:59:33.976368 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 24 05:59:33.976368 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:59:33.976368 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:59:33.976368 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:59:33.976368 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:59:33.977583 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:59:33.977583 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:59:33.977583 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:59:33.977583 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:59:33.977583 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:59:33.977583 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:59:33.977583 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:59:33.977583 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:59:33.977583 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:59:33.977583 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:59:33.977583 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 05:59:33.977583 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 24 05:59:33.977583 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 599/1195] 10.222.0.62 () {56 vars in 1096 bytes} [Wed Aug 24 05:59:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:59:33.982223 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc877739-b0f2-4575-9b4f-59d23f8b0910 None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:59:34.038388 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:59:34.082268 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc877739-b0f2-4575-9b4f-59d23f8b0910 None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:59:34.114917 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed1ddca-9be6-49c5-b5fc-1877a69ab53d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:59:34.115512 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed1ddca-9be6-49c5-b5fc-1877a69ab53d None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:59:34.115793 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed1ddca-9be6-49c5-b5fc-1877a69ab53d None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:59:34.116069 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed1ddca-9be6-49c5-b5fc-1877a69ab53d None admin] PATH_INFO: `/v3/roles` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:59:34.116756 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed1ddca-9be6-49c5-b5fc-1877a69ab53d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:59:34.117061 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed1ddca-9be6-49c5-b5fc-1877a69ab53d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:59:34.118425 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed1ddca-9be6-49c5-b5fc-1877a69ab53d None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:59:34.126769 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 597/1196] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Aug 24 05:59:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:59:34.132675 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455b7a5d-6d3c-409a-8023-c9c9077bdf2e None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:59:34.230922 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455b7a5d-6d3c-409a-8023-c9c9077bdf2e None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:59:34.262903 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3897a6f1-57a3-44a4-bb4f-55f664a1d83c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:59:34.263507 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3897a6f1-57a3-44a4-bb4f-55f664a1d83c None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:59:34.263785 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3897a6f1-57a3-44a4-bb4f-55f664a1d83c None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:59:34.264058 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3897a6f1-57a3-44a4-bb4f-55f664a1d83c None admin] PATH_INFO: `/v3/users/41adc23fb59a46d3a235e4ef37effe7e` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:59:34.265165 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3897a6f1-57a3-44a4-bb4f-55f664a1d83c None admin] RBAC: Authorizing `identity:get_user(user_id=41adc23fb59a46d3a235e4ef37effe7e)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:59:34.266584 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3897a6f1-57a3-44a4-bb4f-55f664a1d83c None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:59:34.268125 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 600/1197] 10.222.0.62 () {56 vars in 1177 bytes} [Wed Aug 24 05:59:34 2022] GET /identity/v3/users/41adc23fb59a46d3a235e4ef37effe7e => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:59:34.273888 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed1ddca-9be6-49c5-b5fc-1877a69ab53d None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:59:34.372372 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed1ddca-9be6-49c5-b5fc-1877a69ab53d None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:59:34.391144 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:59:34.405074 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273d535d-a572-4d04-a6be-5349ad18b142 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:59:34.407992 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273d535d-a572-4d04-a6be-5349ad18b142 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:59:34.407992 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273d535d-a572-4d04-a6be-5349ad18b142 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:59:34.407992 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273d535d-a572-4d04-a6be-5349ad18b142 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:59:34.470857 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273d535d-a572-4d04-a6be-5349ad18b142 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:59:34.472291 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273d535d-a572-4d04-a6be-5349ad18b142 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:59:34.566581 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 598/1198] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Aug 24 05:59:34 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 05:59:34.572447 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3897a6f1-57a3-44a4-bb4f-55f664a1d83c None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:59:34.594266 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:59:34.668180 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3897a6f1-57a3-44a4-bb4f-55f664a1d83c None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:59:34.701594 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156b9aea-6366-40fd-8c93-eb50203b18ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:59:34.702216 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156b9aea-6366-40fd-8c93-eb50203b18ca None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:59:34.702491 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156b9aea-6366-40fd-8c93-eb50203b18ca None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:59:34.702765 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156b9aea-6366-40fd-8c93-eb50203b18ca None admin] PATH_INFO: `/v3/projects/61dd43d6ff6b4bdabc5e3cb34b2407ef` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:59:34.703858 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156b9aea-6366-40fd-8c93-eb50203b18ca None admin] RBAC: Authorizing `identity:get_project(project_id=61dd43d6ff6b4bdabc5e3cb34b2407ef)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:59:34.705309 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156b9aea-6366-40fd-8c93-eb50203b18ca None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:59:34.706962 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 601/1199] 10.222.0.62 () {56 vars in 1186 bytes} [Wed Aug 24 05:59:34 2022] GET /identity/v3/projects/61dd43d6ff6b4bdabc5e3cb34b2407ef => generated 361 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:59:34.712969 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273d535d-a572-4d04-a6be-5349ad18b142 None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:59:34.733675 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:59:34.810028 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273d535d-a572-4d04-a6be-5349ad18b142 None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:59:34.841794 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9a8c43-5b03-4d4c-a5f9-2f8e07fd6fce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:59:34.842379 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9a8c43-5b03-4d4c-a5f9-2f8e07fd6fce None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:59:34.842652 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9a8c43-5b03-4d4c-a5f9-2f8e07fd6fce None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:59:34.842919 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9a8c43-5b03-4d4c-a5f9-2f8e07fd6fce None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:59:34.843617 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9a8c43-5b03-4d4c-a5f9-2f8e07fd6fce None admin] RBAC: Adding query filter params (role.id=49e47cd157d743f6905f1db07ec878a7, scope.project.id=61dd43d6ff6b4bdabc5e3cb34b2407ef, user.id=41adc23fb59a46d3a235e4ef37effe7e) {{(pid=81851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:59:34.843918 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9a8c43-5b03-4d4c-a5f9-2f8e07fd6fce None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:59:34.845381 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9a8c43-5b03-4d4c-a5f9-2f8e07fd6fce None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:59:34.850855 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 599/1200] 10.222.0.62 () {56 vars in 1374 bytes} [Wed Aug 24 05:59:34 2022] GET /identity/v3/role_assignments?user.id=41adc23fb59a46d3a235e4ef37effe7e&scope.project.id=61dd43d6ff6b4bdabc5e3cb34b2407ef&role.id=49e47cd157d743f6905f1db07ec878a7 => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:59:44.697672 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=56 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:59:44.848366 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=58 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:00:51.238573 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec3a999-ecd1-4216-b028-ab56e939b7e1 None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:00:51.239161 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec3a999-ecd1-4216-b028-ab56e939b7e1 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:00:51.239161 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec3a999-ecd1-4216-b028-ab56e939b7e1 None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:00:51.240614 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 602/1201] 10.222.0.62 () {54 vars in 927 bytes} [Wed Aug 24 06:00:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:00:51.245717 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ce1413-16e1-4ba6-b9df-e68d3df1c284 None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:00:51.246146 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ce1413-16e1-4ba6-b9df-e68d3df1c284 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:00:51.246245 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ce1413-16e1-4ba6-b9df-e68d3df1c284 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:00:51.265175 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-a7ce1413-16e1-4ba6-b9df-e68d3df1c284 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:00:51.266401 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-a7ce1413-16e1-4ba6-b9df-e68d3df1c284 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:00:51.331959 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 600/1202] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Aug 24 06:00:51 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 06:00:51.601068 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec3a999-ecd1-4216-b028-ab56e939b7e1 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:00:51.699782 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec3a999-ecd1-4216-b028-ab56e939b7e1 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:00:51.733583 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613664c5-128b-4d59-83d2-2478df9c8228 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:00:51.734210 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613664c5-128b-4d59-83d2-2478df9c8228 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:00:51.734482 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613664c5-128b-4d59-83d2-2478df9c8228 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:00:51.734753 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613664c5-128b-4d59-83d2-2478df9c8228 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:00:51.747784 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613664c5-128b-4d59-83d2-2478df9c8228 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:00:51.749271 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613664c5-128b-4d59-83d2-2478df9c8228 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:00:51.763634 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-613664c5-128b-4d59-83d2-2478df9c8228 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:00:51.763634 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:00:51.763634 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:00:51.763634 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:00:51.763634 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:00:51.763634 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:00:51.763634 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:00:51.763634 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:00:51.763634 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:00:51.763634 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:00:51.763634 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:00:51.763634 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:00:51.763634 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 06:00:51.763634 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 06:00:51.763634 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 06:00:51.763634 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 06:00:51.764904 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:00:51.764904 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:00:51.764904 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:00:51.764904 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:00:51.764904 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:00:51.764904 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:00:51.764904 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:00:51.764904 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 24 06:00:51.764904 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:00:51.764904 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:00:51.764904 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:00:51.764904 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:00:51.764904 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:00:51.764904 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:00:51.764904 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:00:51.764904 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:00:51.764904 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 06:00:51.766067 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 06:00:51.766067 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:00:51.766067 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 24 06:00:51.766067 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 603/1203] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Aug 24 06:00:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:00:51.770794 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ce1413-16e1-4ba6-b9df-e68d3df1c284 None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:00:51.867337 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ce1413-16e1-4ba6-b9df-e68d3df1c284 None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:00:51.894911 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:00:51.900587 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235b46da-2f69-4f6c-8373-9c809ae9c1e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:00:51.902851 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235b46da-2f69-4f6c-8373-9c809ae9c1e0 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:00:51.902851 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235b46da-2f69-4f6c-8373-9c809ae9c1e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:00:51.902851 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235b46da-2f69-4f6c-8373-9c809ae9c1e0 None admin] PATH_INFO: `/v3/domains` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:00:51.902851 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235b46da-2f69-4f6c-8373-9c809ae9c1e0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:00:51.902851 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235b46da-2f69-4f6c-8373-9c809ae9c1e0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:00:51.904530 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235b46da-2f69-4f6c-8373-9c809ae9c1e0 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:00:51.915264 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 601/1204] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Aug 24 06:00:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:00:51.921579 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613664c5-128b-4d59-83d2-2478df9c8228 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:00:51.976166 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:00:52.021648 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613664c5-128b-4d59-83d2-2478df9c8228 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:00:52.054374 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb6f72f-d441-4c8b-b6fe-0333caae900d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:00:52.054977 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb6f72f-d441-4c8b-b6fe-0333caae900d None admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:00:52.055253 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb6f72f-d441-4c8b-b6fe-0333caae900d None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:00:52.055526 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb6f72f-d441-4c8b-b6fe-0333caae900d None admin] PATH_INFO: `/v3/users` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:00:52.056161 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb6f72f-d441-4c8b-b6fe-0333caae900d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:00:52.057620 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb6f72f-d441-4c8b-b6fe-0333caae900d None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:00:52.142234 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 604/1205] 10.222.0.62 () {60 vars in 1132 bytes} [Wed Aug 24 06:00:51 2022] POST /identity/v3/users => generated 251 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:00:54.077580 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d777d039-daa7-4e04-9e1b-222e5f490a87 None None] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:00:54.078131 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d777d039-daa7-4e04-9e1b-222e5f490a87 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:00:54.078131 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d777d039-daa7-4e04-9e1b-222e5f490a87 None None] PATH_INFO: `/` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:00:54.079392 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 602/1206] 10.222.0.62 () {54 vars in 927 bytes} [Wed Aug 24 06:00:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:00:54.084708 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b51053-4067-483c-978a-521c8219b512 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:00:54.085166 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b51053-4067-483c-978a-521c8219b512 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:00:54.085263 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b51053-4067-483c-978a-521c8219b512 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:00:54.103143 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-26b51053-4067-483c-978a-521c8219b512 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:00:54.104315 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-26b51053-4067-483c-978a-521c8219b512 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:00:54.169699 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 605/1207] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Aug 24 06:00:54 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 06:00:54.441498 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d777d039-daa7-4e04-9e1b-222e5f490a87 None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:00:54.542534 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d777d039-daa7-4e04-9e1b-222e5f490a87 None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:00:54.577276 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ed3813-9be5-46da-9c00-2baf262c02a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:00:54.577898 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ed3813-9be5-46da-9c00-2baf262c02a1 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:00:54.578179 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ed3813-9be5-46da-9c00-2baf262c02a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:00:54.578454 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ed3813-9be5-46da-9c00-2baf262c02a1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:00:54.588370 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ed3813-9be5-46da-9c00-2baf262c02a1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 06:00:54.593721 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application [None req-55ed3813-9be5-46da-9c00-2baf262c02a1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 06:00:54.593721 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:00:54.593721 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:00:54.593721 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:00:54.593721 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:00:54.593721 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:00:54.593721 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:00:54.593721 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:00:54.593721 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:00:54.593721 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:00:54.593721 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:00:54.593721 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:00:54.593721 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 06:00:54.593721 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 06:00:54.593721 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 06:00:54.593721 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 06:00:54.594802 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 06:00:54.594802 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 06:00:54.594802 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 06:00:54.594802 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 06:00:54.594802 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:00:54.594802 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:00:54.594802 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:00:54.594802 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:00:54.594802 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:00:54.594802 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:00:54.594802 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:00:54.594802 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application with Lock( Aug 24 06:00:54.594802 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:00:54.594802 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:00:54.594802 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:00:54.594802 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:00:54.595858 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:00:54.595858 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:00:54.595858 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:00:54.595858 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:00:54.595858 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 06:00:54.595858 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 06:00:54.595858 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 06:00:54.595858 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 06:00:54.595858 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 06:00:54.595858 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 06:00:54.595858 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 06:00:54.595858 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Aug 24 06:00:54.595858 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 603/1208] 10.222.0.62 () {56 vars in 1102 bytes} [Wed Aug 24 06:00:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:00:54.601220 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b51053-4067-483c-978a-521c8219b512 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:00:54.639805 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:00:54.698454 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b51053-4067-483c-978a-521c8219b512 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:00:54.732267 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4049e19-bfd0-4123-ac35-3be2ac0c10ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:00:54.732896 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4049e19-bfd0-4123-ac35-3be2ac0c10ec None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:00:54.733173 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4049e19-bfd0-4123-ac35-3be2ac0c10ec None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:00:54.733442 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4049e19-bfd0-4123-ac35-3be2ac0c10ec None admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:00:54.734088 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4049e19-bfd0-4123-ac35-3be2ac0c10ec None admin] RBAC: Adding query filter params (name=service) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:00:54.734386 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4049e19-bfd0-4123-ac35-3be2ac0c10ec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:00:54.735732 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4049e19-bfd0-4123-ac35-3be2ac0c10ec None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:00:54.743165 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 606/1209] 10.222.0.62 () {56 vars in 1103 bytes} [Wed Aug 24 06:00:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:00:54.748914 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ed3813-9be5-46da-9c00-2baf262c02a1 None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:00:54.799615 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:00:54.847686 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ed3813-9be5-46da-9c00-2baf262c02a1 None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:00:54.880843 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0fde16-a219-42e1-9b33-766505ed8783 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:00:54.881469 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0fde16-a219-42e1-9b33-766505ed8783 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:00:54.881759 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0fde16-a219-42e1-9b33-766505ed8783 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:00:54.882066 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0fde16-a219-42e1-9b33-766505ed8783 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:00:54.892848 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0fde16-a219-42e1-9b33-766505ed8783 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:00:54.894320 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0fde16-a219-42e1-9b33-766505ed8783 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:00:54.908518 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application [None req-5e0fde16-a219-42e1-9b33-766505ed8783 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:00:54.908518 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:00:54.908518 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:00:54.908518 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:00:54.908518 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:00:54.908518 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:00:54.908518 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:00:54.908518 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:00:54.908518 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:00:54.908518 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:00:54.908518 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:00:54.908518 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:00:54.908518 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 06:00:54.908518 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 06:00:54.908518 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 06:00:54.908518 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 06:00:54.910092 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:00:54.910092 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:00:54.910092 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:00:54.910092 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:00:54.910092 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:00:54.910092 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:00:54.910092 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:00:54.910092 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application with Lock( Aug 24 06:00:54.910092 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:00:54.910092 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:00:54.910092 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:00:54.910092 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:00:54.910092 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:00:54.910092 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:00:54.910092 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:00:54.910092 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:00:54.910092 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 06:00:54.911132 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 06:00:54.911132 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:00:54.911132 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Aug 24 06:00:54.911132 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 604/1210] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Aug 24 06:00:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:00:54.915574 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4049e19-bfd0-4123-ac35-3be2ac0c10ec None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:00:55.010970 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:00:55.014339 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4049e19-bfd0-4123-ac35-3be2ac0c10ec None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:00:55.047150 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b74703-6501-4c3e-b8de-531c825ab25e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:00:55.047745 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b74703-6501-4c3e-b8de-531c825ab25e None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:00:55.048024 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b74703-6501-4c3e-b8de-531c825ab25e None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:00:55.048299 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b74703-6501-4c3e-b8de-531c825ab25e None admin] PATH_INFO: `/v3/domains` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:00:55.048972 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b74703-6501-4c3e-b8de-531c825ab25e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:00:55.049291 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b74703-6501-4c3e-b8de-531c825ab25e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:00:55.050643 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b74703-6501-4c3e-b8de-531c825ab25e None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:00:55.061467 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 607/1211] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Aug 24 06:00:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:00:55.067176 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0fde16-a219-42e1-9b33-766505ed8783 None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:00:55.165523 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0fde16-a219-42e1-9b33-766505ed8783 None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:00:55.167046 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:00:55.198630 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6338f45d-9e59-4df3-9b15-8ec3ef971ca3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:00:55.199277 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6338f45d-9e59-4df3-9b15-8ec3ef971ca3 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:00:55.199548 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6338f45d-9e59-4df3-9b15-8ec3ef971ca3 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:00:55.201352 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6338f45d-9e59-4df3-9b15-8ec3ef971ca3 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:00:55.207478 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6338f45d-9e59-4df3-9b15-8ec3ef971ca3 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:00:55.208950 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6338f45d-9e59-4df3-9b15-8ec3ef971ca3 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:00:55.220612 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application [None req-6338f45d-9e59-4df3-9b15-8ec3ef971ca3 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 24 06:00:55.220612 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:00:55.220612 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:00:55.220612 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:00:55.220612 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:00:55.220612 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:00:55.220612 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:00:55.220612 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:00:55.220612 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:00:55.220612 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:00:55.220612 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:00:55.220612 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:00:55.220612 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 06:00:55.220612 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 06:00:55.220612 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 06:00:55.220612 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 06:00:55.221754 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:00:55.221754 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:00:55.221754 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 06:00:55.221754 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 06:00:55.221754 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 06:00:55.221754 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 06:00:55.221754 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:00:55.221754 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:00:55.221754 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:00:55.221754 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:00:55.221754 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:00:55.221754 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application with Lock( Aug 24 06:00:55.221754 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:00:55.221754 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:00:55.221754 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:00:55.221754 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:00:55.221754 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:00:55.221754 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:00:55.222751 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:00:55.222751 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:00:55.222751 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 06:00:55.222751 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 06:00:55.222751 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 06:00:55.222751 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 06:00:55.222751 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 06:00:55.222751 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 06:00:55.222751 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 24 06:00:55.222751 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Aug 24 06:00:55.222751 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 605/1212] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Aug 24 06:00:55 2022] GET /identity/v3/users/placement => generated 87 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:00:55.227521 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b74703-6501-4c3e-b8de-531c825ab25e None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:00:55.324412 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b74703-6501-4c3e-b8de-531c825ab25e None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:00:55.356364 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a75220c-08c3-4dab-943d-b6b06f157ce1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:00:55.357305 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a75220c-08c3-4dab-943d-b6b06f157ce1 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:00:55.357305 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a75220c-08c3-4dab-943d-b6b06f157ce1 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:00:55.357578 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a75220c-08c3-4dab-943d-b6b06f157ce1 None admin] PATH_INFO: `/v3/users` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:00:55.358288 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a75220c-08c3-4dab-943d-b6b06f157ce1 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:00:55.358609 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a75220c-08c3-4dab-943d-b6b06f157ce1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:00:55.360016 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a75220c-08c3-4dab-943d-b6b06f157ce1 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:00:55.376836 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 608/1213] 10.222.0.62 () {56 vars in 1143 bytes} [Wed Aug 24 06:00:55 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:00:55.382655 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6338f45d-9e59-4df3-9b15-8ec3ef971ca3 None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:00:55.480830 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6338f45d-9e59-4df3-9b15-8ec3ef971ca3 None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:00:55.515580 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30a9040-dd44-44dc-856f-5c2158646aa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:00:55.516180 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30a9040-dd44-44dc-856f-5c2158646aa0 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:00:55.516466 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30a9040-dd44-44dc-856f-5c2158646aa0 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:00:55.516777 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30a9040-dd44-44dc-856f-5c2158646aa0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:00:55.546493 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:00:55.584877 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30a9040-dd44-44dc-856f-5c2158646aa0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:00:55.586292 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30a9040-dd44-44dc-856f-5c2158646aa0 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:00:55.682040 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 606/1214] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Aug 24 06:00:55 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 24 06:00:55.688049 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a75220c-08c3-4dab-943d-b6b06f157ce1 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:00:55.707504 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:00:55.785702 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a75220c-08c3-4dab-943d-b6b06f157ce1 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:00:55.818101 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22ad4fe-6f81-4983-b611-2301d1655ccd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:00:55.818752 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22ad4fe-6f81-4983-b611-2301d1655ccd None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:00:55.819029 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22ad4fe-6f81-4983-b611-2301d1655ccd None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:00:55.819297 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22ad4fe-6f81-4983-b611-2301d1655ccd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:00:55.830333 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22ad4fe-6f81-4983-b611-2301d1655ccd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:00:55.832900 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22ad4fe-6f81-4983-b611-2301d1655ccd None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:00:55.846580 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-d22ad4fe-6f81-4983-b611-2301d1655ccd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:00:55.846580 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:00:55.846580 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:00:55.846580 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:00:55.846580 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:00:55.846580 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:00:55.846580 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:00:55.846580 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:00:55.846580 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:00:55.846580 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:00:55.846580 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:00:55.846580 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:00:55.846580 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 06:00:55.846580 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 06:00:55.846580 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 06:00:55.846580 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 06:00:55.847663 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:00:55.847663 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:00:55.847663 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:00:55.847663 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:00:55.847663 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:00:55.847663 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:00:55.847663 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:00:55.847663 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 24 06:00:55.847663 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:00:55.847663 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:00:55.847663 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:00:55.847663 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:00:55.847663 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:00:55.847663 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:00:55.847663 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:00:55.847663 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:00:55.847663 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 06:00:55.848993 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 06:00:55.848993 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:00:55.848993 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 24 06:00:55.848993 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 609/1215] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Aug 24 06:00:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:00:55.853360 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30a9040-dd44-44dc-856f-5c2158646aa0 None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:00:55.913395 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:00:55.949669 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30a9040-dd44-44dc-856f-5c2158646aa0 None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:00:55.981947 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d012e5-5cd3-4991-8b45-0225da9fc69e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:00:55.982539 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d012e5-5cd3-4991-8b45-0225da9fc69e None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:00:55.982814 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d012e5-5cd3-4991-8b45-0225da9fc69e None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:00:55.983081 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d012e5-5cd3-4991-8b45-0225da9fc69e None admin] PATH_INFO: `/v3/domains` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:00:55.983698 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d012e5-5cd3-4991-8b45-0225da9fc69e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:00:55.984001 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d012e5-5cd3-4991-8b45-0225da9fc69e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:00:55.985369 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d012e5-5cd3-4991-8b45-0225da9fc69e None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:00:55.996480 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 607/1216] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Aug 24 06:00:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:00:56.002975 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22ad4fe-6f81-4983-b611-2301d1655ccd None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:00:56.070444 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:00:56.099910 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22ad4fe-6f81-4983-b611-2301d1655ccd None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:00:56.131415 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fda036b-0cad-4310-957c-3f8b605438cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:00:56.132234 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fda036b-0cad-4310-957c-3f8b605438cb None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:00:56.132727 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fda036b-0cad-4310-957c-3f8b605438cb None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:00:56.133420 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fda036b-0cad-4310-957c-3f8b605438cb None admin] PATH_INFO: `/v3/projects/service` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:00:56.140705 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fda036b-0cad-4310-957c-3f8b605438cb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:00:56.142136 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fda036b-0cad-4310-957c-3f8b605438cb None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:00:56.153054 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-9fda036b-0cad-4310-957c-3f8b605438cb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 06:00:56.153054 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:00:56.153054 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:00:56.153054 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:00:56.153054 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:00:56.153054 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:00:56.153054 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:00:56.153054 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:00:56.153054 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:00:56.153054 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:00:56.153054 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:00:56.153054 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:00:56.153054 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 06:00:56.153054 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 06:00:56.153054 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 06:00:56.154007 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 06:00:56.154007 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:00:56.154007 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:00:56.154007 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:00:56.154007 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:00:56.154007 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:00:56.154007 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:00:56.154007 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:00:56.154007 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 24 06:00:56.154007 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:00:56.154007 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:00:56.154007 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:00:56.154007 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:00:56.154007 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:00:56.154007 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:00:56.154007 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:00:56.154007 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:00:56.154952 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 06:00:56.154952 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 06:00:56.154952 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 06:00:56.154952 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 06:00:56.154952 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 06:00:56.154952 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 06:00:56.154952 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 06:00:56.154952 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 24 06:00:56.154952 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 610/1217] 10.222.0.62 () {56 vars in 1111 bytes} [Wed Aug 24 06:00:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:00:56.159892 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d012e5-5cd3-4991-8b45-0225da9fc69e None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:00:56.255806 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d012e5-5cd3-4991-8b45-0225da9fc69e None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:00:56.277946 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:00:56.288949 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baaace12-b78b-4a96-bf67-caa66aa5c298 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:00:56.289541 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaace12-b78b-4a96-bf67-caa66aa5c298 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:00:56.289816 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaace12-b78b-4a96-bf67-caa66aa5c298 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:00:56.290087 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaace12-b78b-4a96-bf67-caa66aa5c298 None admin] PATH_INFO: `/v3/projects` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:00:56.290764 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baaace12-b78b-4a96-bf67-caa66aa5c298 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:00:56.291063 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baaace12-b78b-4a96-bf67-caa66aa5c298 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:00:56.292471 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baaace12-b78b-4a96-bf67-caa66aa5c298 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:00:56.303806 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 608/1218] 10.222.0.62 () {56 vars in 1148 bytes} [Wed Aug 24 06:00:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:00:56.310640 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fda036b-0cad-4310-957c-3f8b605438cb None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:00:56.406782 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fda036b-0cad-4310-957c-3f8b605438cb None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:00:56.435133 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:00:56.440807 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1120898-e35b-4baa-bda6-f740ac6533e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:00:56.442561 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1120898-e35b-4baa-bda6-f740ac6533e1 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:00:56.442561 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1120898-e35b-4baa-bda6-f740ac6533e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:00:56.442561 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1120898-e35b-4baa-bda6-f740ac6533e1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:00:56.442881 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1120898-e35b-4baa-bda6-f740ac6533e1 None admin] RBAC: Adding query filter params (role.id=5d95a5fc0f324a92905533aa8d34ac15, scope.project.id=181f4762037e454489d9552cc603179c, user.id=abe9d8c9291845dfa4ef55034e5c9e32) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:00:56.442881 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1120898-e35b-4baa-bda6-f740ac6533e1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:00:56.444304 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1120898-e35b-4baa-bda6-f740ac6533e1 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:00:56.449457 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 611/1219] 10.222.0.62 () {56 vars in 1374 bytes} [Wed Aug 24 06:00:56 2022] GET /identity/v3/role_assignments?user.id=abe9d8c9291845dfa4ef55034e5c9e32&scope.project.id=181f4762037e454489d9552cc603179c&role.id=5d95a5fc0f324a92905533aa8d34ac15 => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:00:58.297057 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e822cc-8057-4c27-a526-fe658341ccfd None None] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:00:58.297652 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e822cc-8057-4c27-a526-fe658341ccfd None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:00:58.297652 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e822cc-8057-4c27-a526-fe658341ccfd None None] PATH_INFO: `/` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:00:58.298937 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 609/1220] 10.222.0.62 () {54 vars in 927 bytes} [Wed Aug 24 06:00:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:00:58.303898 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699e07d2-9e6e-488a-b0ae-c108aadc7df0 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:00:58.304348 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699e07d2-9e6e-488a-b0ae-c108aadc7df0 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:00:58.304522 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699e07d2-9e6e-488a-b0ae-c108aadc7df0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:00:58.322663 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-699e07d2-9e6e-488a-b0ae-c108aadc7df0 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:00:58.330006 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-699e07d2-9e6e-488a-b0ae-c108aadc7df0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:00:58.388716 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 612/1221] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Aug 24 06:00:58 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 06:00:58.654550 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e822cc-8057-4c27-a526-fe658341ccfd None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:00:58.755299 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e822cc-8057-4c27-a526-fe658341ccfd None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:00:58.787304 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cb3d00-a102-44be-9fd5-ee04d794d2d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:00:58.787917 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cb3d00-a102-44be-9fd5-ee04d794d2d3 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:00:58.788196 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cb3d00-a102-44be-9fd5-ee04d794d2d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:00:58.788467 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cb3d00-a102-44be-9fd5-ee04d794d2d3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:00:58.799081 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3cb3d00-a102-44be-9fd5-ee04d794d2d3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 06:00:58.803727 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application [None req-c3cb3d00-a102-44be-9fd5-ee04d794d2d3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 06:00:58.803727 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:00:58.803727 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:00:58.803727 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:00:58.803727 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:00:58.803727 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:00:58.803727 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:00:58.803727 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:00:58.803727 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:00:58.803727 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:00:58.803727 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:00:58.803727 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:00:58.803727 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 06:00:58.803727 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 06:00:58.803727 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 06:00:58.803727 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 06:00:58.808383 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 06:00:58.808383 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 06:00:58.808383 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 06:00:58.808383 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 06:00:58.808383 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:00:58.808383 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:00:58.808383 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:00:58.808383 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:00:58.808383 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:00:58.808383 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:00:58.808383 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:00:58.808383 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application with Lock( Aug 24 06:00:58.808383 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:00:58.808383 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:00:58.808383 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:00:58.808383 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:00:58.810138 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:00:58.810138 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:00:58.810138 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:00:58.810138 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:00:58.810138 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 06:00:58.810138 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 06:00:58.810138 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 06:00:58.810138 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 06:00:58.810138 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 06:00:58.810138 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 06:00:58.810138 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 06:00:58.810138 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Aug 24 06:00:58.810138 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 610/1222] 10.222.0.62 () {56 vars in 1102 bytes} [Wed Aug 24 06:00:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 06:00:58.811613 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699e07d2-9e6e-488a-b0ae-c108aadc7df0 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:00:58.908308 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699e07d2-9e6e-488a-b0ae-c108aadc7df0 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:00:58.931737 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:00:58.942712 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfa814d-0fc8-424e-b20e-3813f07e1ddd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:00:58.945182 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfa814d-0fc8-424e-b20e-3813f07e1ddd None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:00:58.945182 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfa814d-0fc8-424e-b20e-3813f07e1ddd None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:00:58.945182 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfa814d-0fc8-424e-b20e-3813f07e1ddd None admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:00:58.945182 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfa814d-0fc8-424e-b20e-3813f07e1ddd None admin] RBAC: Adding query filter params (name=service) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:00:58.945182 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfa814d-0fc8-424e-b20e-3813f07e1ddd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:00:58.946186 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfa814d-0fc8-424e-b20e-3813f07e1ddd None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:00:58.953882 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 613/1223] 10.222.0.62 () {56 vars in 1103 bytes} [Wed Aug 24 06:00:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:00:58.959635 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cb3d00-a102-44be-9fd5-ee04d794d2d3 None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:00:59.004094 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:00:59.058575 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cb3d00-a102-44be-9fd5-ee04d794d2d3 None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:00:59.092916 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbca753e-a810-45a4-b7a8-8503ba7e3782 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:00:59.093594 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbca753e-a810-45a4-b7a8-8503ba7e3782 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:00:59.093868 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbca753e-a810-45a4-b7a8-8503ba7e3782 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:00:59.094136 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbca753e-a810-45a4-b7a8-8503ba7e3782 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:00:59.103805 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbca753e-a810-45a4-b7a8-8503ba7e3782 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:00:59.105355 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbca753e-a810-45a4-b7a8-8503ba7e3782 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:00:59.119907 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application [None req-fbca753e-a810-45a4-b7a8-8503ba7e3782 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:00:59.119907 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:00:59.119907 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:00:59.119907 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:00:59.119907 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:00:59.119907 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:00:59.119907 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:00:59.119907 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:00:59.119907 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:00:59.119907 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:00:59.119907 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:00:59.119907 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:00:59.119907 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 06:00:59.119907 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 06:00:59.119907 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 06:00:59.119907 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 06:00:59.121062 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:00:59.121062 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:00:59.121062 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:00:59.121062 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:00:59.121062 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:00:59.121062 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:00:59.121062 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:00:59.121062 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application with Lock( Aug 24 06:00:59.121062 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:00:59.121062 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:00:59.121062 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:00:59.121062 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:00:59.121062 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:00:59.121062 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:00:59.121062 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:00:59.121062 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:00:59.121062 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 06:00:59.122191 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 06:00:59.122191 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:00:59.122191 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Aug 24 06:00:59.122191 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 611/1224] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Aug 24 06:00:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:00:59.126837 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfa814d-0fc8-424e-b20e-3813f07e1ddd None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:00:59.225777 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfa814d-0fc8-424e-b20e-3813f07e1ddd None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:00:59.257950 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567879f3-d774-413f-8897-7ceee082bd58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:00:59.258541 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567879f3-d774-413f-8897-7ceee082bd58 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:00:59.259638 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567879f3-d774-413f-8897-7ceee082bd58 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:00:59.259638 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567879f3-d774-413f-8897-7ceee082bd58 None admin] PATH_INFO: `/v3/domains` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:00:59.259802 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567879f3-d774-413f-8897-7ceee082bd58 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:00:59.260035 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567879f3-d774-413f-8897-7ceee082bd58 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:00:59.261417 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567879f3-d774-413f-8897-7ceee082bd58 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:00:59.272664 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 614/1225] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Aug 24 06:00:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:00:59.278964 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbca753e-a810-45a4-b7a8-8503ba7e3782 None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:00:59.370838 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:00:59.377322 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbca753e-a810-45a4-b7a8-8503ba7e3782 None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:00:59.411194 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c4a64e-9ac4-4d72-b46f-35c95eb61a50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:00:59.411817 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c4a64e-9ac4-4d72-b46f-35c95eb61a50 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:00:59.412101 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c4a64e-9ac4-4d72-b46f-35c95eb61a50 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:00:59.412375 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c4a64e-9ac4-4d72-b46f-35c95eb61a50 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:00:59.419522 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c4a64e-9ac4-4d72-b46f-35c95eb61a50 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:00:59.421030 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c4a64e-9ac4-4d72-b46f-35c95eb61a50 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:00:59.432182 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application [None req-62c4a64e-9ac4-4d72-b46f-35c95eb61a50 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 24 06:00:59.432182 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:00:59.432182 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:00:59.432182 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:00:59.432182 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:00:59.432182 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:00:59.432182 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:00:59.432182 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:00:59.432182 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:00:59.432182 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:00:59.432182 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:00:59.432182 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:00:59.432182 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 06:00:59.432182 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 06:00:59.432182 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 06:00:59.432182 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 06:00:59.433367 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:00:59.433367 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:00:59.433367 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 06:00:59.433367 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 06:00:59.433367 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 06:00:59.433367 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 06:00:59.433367 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:00:59.433367 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:00:59.433367 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:00:59.433367 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:00:59.433367 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:00:59.433367 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application with Lock( Aug 24 06:00:59.433367 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:00:59.433367 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:00:59.433367 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:00:59.433367 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:00:59.433367 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:00:59.433367 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:00:59.434766 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:00:59.434766 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:00:59.434766 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 06:00:59.434766 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 06:00:59.434766 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 06:00:59.434766 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 06:00:59.434766 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 06:00:59.434766 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 06:00:59.434766 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 24 06:00:59.434766 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Aug 24 06:00:59.434766 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 612/1226] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Aug 24 06:00:59 2022] GET /identity/v3/users/placement => generated 87 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:00:59.439186 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567879f3-d774-413f-8897-7ceee082bd58 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:00:59.486182 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:00:59.538016 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567879f3-d774-413f-8897-7ceee082bd58 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:00:59.570868 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccde9e0b-e2cf-4d80-afe4-ba3410c908f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:00:59.571450 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccde9e0b-e2cf-4d80-afe4-ba3410c908f5 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:00:59.571727 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccde9e0b-e2cf-4d80-afe4-ba3410c908f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:00:59.571997 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccde9e0b-e2cf-4d80-afe4-ba3410c908f5 None admin] PATH_INFO: `/v3/users` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:00:59.572772 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccde9e0b-e2cf-4d80-afe4-ba3410c908f5 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:00:59.573077 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccde9e0b-e2cf-4d80-afe4-ba3410c908f5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:00:59.575129 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccde9e0b-e2cf-4d80-afe4-ba3410c908f5 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:00:59.592517 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 615/1227] 10.222.0.62 () {56 vars in 1143 bytes} [Wed Aug 24 06:00:59 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:00:59.598260 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c4a64e-9ac4-4d72-b46f-35c95eb61a50 None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:00:59.697236 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c4a64e-9ac4-4d72-b46f-35c95eb61a50 None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:00:59.731069 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6a86ce-978a-4a1a-9360-50c8d84faa04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:00:59.731687 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6a86ce-978a-4a1a-9360-50c8d84faa04 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:00:59.731959 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6a86ce-978a-4a1a-9360-50c8d84faa04 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:00:59.732224 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6a86ce-978a-4a1a-9360-50c8d84faa04 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:00:59.742418 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6a86ce-978a-4a1a-9360-50c8d84faa04 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:00:59.743783 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6a86ce-978a-4a1a-9360-50c8d84faa04 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:00:59.757807 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application [None req-0d6a86ce-978a-4a1a-9360-50c8d84faa04 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:00:59.757807 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:00:59.757807 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:00:59.757807 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:00:59.757807 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:00:59.757807 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:00:59.757807 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:00:59.757807 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:00:59.757807 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:00:59.757807 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:00:59.757807 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:00:59.757807 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:00:59.757807 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 06:00:59.757807 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 06:00:59.757807 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 06:00:59.757807 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 06:00:59.758961 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:00:59.758961 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:00:59.758961 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:00:59.758961 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:00:59.758961 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:00:59.758961 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:00:59.758961 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:00:59.758961 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application with Lock( Aug 24 06:00:59.758961 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:00:59.758961 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:00:59.758961 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:00:59.758961 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:00:59.758961 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:00:59.758961 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:00:59.758961 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:00:59.758961 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:00:59.758961 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 06:00:59.760272 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 06:00:59.760272 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:00:59.760272 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Aug 24 06:00:59.760272 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 613/1228] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Aug 24 06:00:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:00:59.765383 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccde9e0b-e2cf-4d80-afe4-ba3410c908f5 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:00:59.862938 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccde9e0b-e2cf-4d80-afe4-ba3410c908f5 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:00:59.865218 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:00:59.896125 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbb6410-fa5b-4972-b826-719a8c6d989d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:00:59.896756 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbb6410-fa5b-4972-b826-719a8c6d989d None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:00:59.897031 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbb6410-fa5b-4972-b826-719a8c6d989d None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:00:59.897296 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbb6410-fa5b-4972-b826-719a8c6d989d None admin] PATH_INFO: `/v3/domains` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:00:59.897920 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dbb6410-fa5b-4972-b826-719a8c6d989d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:00:59.898217 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dbb6410-fa5b-4972-b826-719a8c6d989d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:00:59.899555 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dbb6410-fa5b-4972-b826-719a8c6d989d None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:00:59.910599 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 616/1229] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Aug 24 06:00:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:00:59.916273 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6a86ce-978a-4a1a-9360-50c8d84faa04 None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:00:59.925802 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:01:00.011874 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6a86ce-978a-4a1a-9360-50c8d84faa04 None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:01:00.043642 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cc2855-2a12-4ca0-8939-617ae133b7fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:01:00.044530 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cc2855-2a12-4ca0-8939-617ae133b7fd None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:01:00.044802 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cc2855-2a12-4ca0-8939-617ae133b7fd None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:01:00.045069 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cc2855-2a12-4ca0-8939-617ae133b7fd None admin] PATH_INFO: `/v3/projects/service` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:01:00.051322 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cc2855-2a12-4ca0-8939-617ae133b7fd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:01:00.052782 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cc2855-2a12-4ca0-8939-617ae133b7fd None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:01:00.063996 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application [None req-96cc2855-2a12-4ca0-8939-617ae133b7fd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 06:01:00.063996 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:01:00.063996 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:01:00.063996 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:01:00.063996 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:01:00.063996 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:01:00.063996 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:01:00.063996 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:01:00.063996 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:01:00.063996 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:01:00.063996 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:01:00.063996 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:01:00.063996 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 06:01:00.063996 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 06:01:00.063996 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 06:01:00.073195 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 06:01:00.073195 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:01:00.073195 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:01:00.073195 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:01:00.073195 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:01:00.073195 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:01:00.073195 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:01:00.073195 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:01:00.073195 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application with Lock( Aug 24 06:01:00.073195 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:01:00.073195 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:01:00.073195 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:01:00.073195 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:01:00.073195 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:01:00.073195 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:01:00.073195 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:01:00.073195 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:01:00.074277 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 06:01:00.074277 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 06:01:00.074277 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 06:01:00.074277 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 06:01:00.074277 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 06:01:00.074277 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 06:01:00.074277 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 06:01:00.074277 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Aug 24 06:01:00.074277 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 614/1230] 10.222.0.62 () {56 vars in 1111 bytes} [Wed Aug 24 06:00:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:01:00.075544 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbb6410-fa5b-4972-b826-719a8c6d989d None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:01:00.177465 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbb6410-fa5b-4972-b826-719a8c6d989d None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:01:00.209938 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda86acc-b348-4cb4-9b4e-d3860f7ac234 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:01:00.210535 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda86acc-b348-4cb4-9b4e-d3860f7ac234 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:01:00.210809 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda86acc-b348-4cb4-9b4e-d3860f7ac234 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:01:00.211078 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda86acc-b348-4cb4-9b4e-d3860f7ac234 None admin] PATH_INFO: `/v3/projects` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:01:00.211743 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda86acc-b348-4cb4-9b4e-d3860f7ac234 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:01:00.212045 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda86acc-b348-4cb4-9b4e-d3860f7ac234 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:01:00.213493 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda86acc-b348-4cb4-9b4e-d3860f7ac234 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:01:00.225033 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 617/1231] 10.222.0.62 () {56 vars in 1148 bytes} [Wed Aug 24 06:01:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:01:00.231521 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cc2855-2a12-4ca0-8939-617ae133b7fd None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:01:00.296278 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:01:00.329175 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cc2855-2a12-4ca0-8939-617ae133b7fd None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:01:00.360742 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ddda4f-c82f-4f3b-817d-6af2b0cf6001 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:01:00.361377 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ddda4f-c82f-4f3b-817d-6af2b0cf6001 None admin] REQUEST_METHOD: `PUT` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:01:00.361652 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ddda4f-c82f-4f3b-817d-6af2b0cf6001 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:01:00.361922 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ddda4f-c82f-4f3b-817d-6af2b0cf6001 None admin] PATH_INFO: `/v3/projects/181f4762037e454489d9552cc603179c/users/abe9d8c9291845dfa4ef55034e5c9e32/roles/5d95a5fc0f324a92905533aa8d34ac15` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:01:00.375840 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ddda4f-c82f-4f3b-817d-6af2b0cf6001 None admin] RBAC: Authorizing `identity:create_grant(project_id=181f4762037e454489d9552cc603179c, user_id=abe9d8c9291845dfa4ef55034e5c9e32, role_id=5d95a5fc0f324a92905533aa8d34ac15)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:01:00.377304 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ddda4f-c82f-4f3b-817d-6af2b0cf6001 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:01:00.489573 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 615/1232] 10.222.0.62 () {58 vars in 1439 bytes} [Wed Aug 24 06:01:00 2022] PUT /identity/v3/projects/181f4762037e454489d9552cc603179c/users/abe9d8c9291845dfa4ef55034e5c9e32/roles/5d95a5fc0f324a92905533aa8d34ac15 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:01:02.379401 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ed940a-56aa-4e63-af31-45251f67da88 None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:01:02.379968 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ed940a-56aa-4e63-af31-45251f67da88 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:01:02.379968 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ed940a-56aa-4e63-af31-45251f67da88 None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:01:02.381421 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 618/1233] 10.222.0.62 () {54 vars in 927 bytes} [Wed Aug 24 06:01:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 06:01:02.386832 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91dcd01-c9aa-4028-b988-28d42590eefd None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:01:02.391409 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91dcd01-c9aa-4028-b988-28d42590eefd None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:01:02.391409 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91dcd01-c9aa-4028-b988-28d42590eefd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:01:02.405105 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-e91dcd01-c9aa-4028-b988-28d42590eefd None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:01:02.406272 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-e91dcd01-c9aa-4028-b988-28d42590eefd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:01:02.471261 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 616/1234] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Aug 24 06:01:02 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 06:01:02.743940 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ed940a-56aa-4e63-af31-45251f67da88 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:01:02.769333 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:01:02.843260 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ed940a-56aa-4e63-af31-45251f67da88 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:01:02.876956 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a834b3d-70fa-4bf1-bf57-10e417421c46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:01:02.877618 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a834b3d-70fa-4bf1-bf57-10e417421c46 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:01:02.877898 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a834b3d-70fa-4bf1-bf57-10e417421c46 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:01:02.878172 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a834b3d-70fa-4bf1-bf57-10e417421c46 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:01:02.887838 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a834b3d-70fa-4bf1-bf57-10e417421c46 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 06:01:02.892538 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-3a834b3d-70fa-4bf1-bf57-10e417421c46 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 06:01:02.892538 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:01:02.892538 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:01:02.892538 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:01:02.892538 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:01:02.892538 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:01:02.892538 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:01:02.892538 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:01:02.892538 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:01:02.892538 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:01:02.892538 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:01:02.892538 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:01:02.892538 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 06:01:02.892538 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 06:01:02.892538 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 06:01:02.892538 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 06:01:02.893598 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 06:01:02.893598 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 06:01:02.893598 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 06:01:02.893598 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 06:01:02.893598 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:01:02.893598 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:01:02.893598 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:01:02.893598 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:01:02.893598 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:01:02.893598 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:01:02.893598 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:01:02.893598 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 24 06:01:02.893598 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:01:02.893598 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:01:02.893598 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:01:02.893598 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:01:02.894526 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:01:02.894526 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:01:02.894526 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:01:02.894526 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:01:02.894526 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 06:01:02.894526 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 06:01:02.894526 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 06:01:02.894526 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 06:01:02.894526 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 06:01:02.894526 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 06:01:02.894526 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 06:01:02.894526 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 24 06:01:02.894526 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 619/1235] 10.222.0.62 () {56 vars in 1102 bytes} [Wed Aug 24 06:01:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:01:02.899675 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91dcd01-c9aa-4028-b988-28d42590eefd None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:01:02.945674 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:01:02.996658 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91dcd01-c9aa-4028-b988-28d42590eefd None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:01:03.030127 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6fd8ce-78d3-47e5-81e3-858fe0e89c01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:01:03.030725 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6fd8ce-78d3-47e5-81e3-858fe0e89c01 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:01:03.031007 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6fd8ce-78d3-47e5-81e3-858fe0e89c01 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:01:03.031279 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6fd8ce-78d3-47e5-81e3-858fe0e89c01 None admin] PATH_INFO: `/v3/roles` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:01:03.031931 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6fd8ce-78d3-47e5-81e3-858fe0e89c01 None admin] RBAC: Adding query filter params (name=service) {{(pid=81851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:01:03.032231 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6fd8ce-78d3-47e5-81e3-858fe0e89c01 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:01:03.033624 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6fd8ce-78d3-47e5-81e3-858fe0e89c01 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:01:03.041307 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 617/1236] 10.222.0.62 () {56 vars in 1103 bytes} [Wed Aug 24 06:01:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 06:01:03.048080 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a834b3d-70fa-4bf1-bf57-10e417421c46 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:01:03.132170 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:01:03.148449 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a834b3d-70fa-4bf1-bf57-10e417421c46 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:01:03.182304 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e024ac-008f-4736-8380-0b933b6ec9bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:01:03.182917 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e024ac-008f-4736-8380-0b933b6ec9bf None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:01:03.183189 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e024ac-008f-4736-8380-0b933b6ec9bf None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:01:03.183453 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e024ac-008f-4736-8380-0b933b6ec9bf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:01:03.193173 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e024ac-008f-4736-8380-0b933b6ec9bf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:01:03.194638 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e024ac-008f-4736-8380-0b933b6ec9bf None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:01:03.219175 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-40e024ac-008f-4736-8380-0b933b6ec9bf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:01:03.219175 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:01:03.219175 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:01:03.219175 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:01:03.219175 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:01:03.219175 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:01:03.219175 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:01:03.219175 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:01:03.219175 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:01:03.219175 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:01:03.219175 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:01:03.219175 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:01:03.219175 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 06:01:03.219175 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 06:01:03.219175 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 06:01:03.219175 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 06:01:03.220334 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:01:03.220334 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:01:03.220334 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:01:03.220334 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:01:03.220334 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:01:03.220334 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:01:03.220334 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:01:03.220334 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 24 06:01:03.220334 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:01:03.220334 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:01:03.220334 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:01:03.220334 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:01:03.220334 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:01:03.220334 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:01:03.220334 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:01:03.220334 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:01:03.220334 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 06:01:03.221652 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 06:01:03.221652 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:01:03.221652 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 24 06:01:03.221652 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 620/1237] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Aug 24 06:01:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:01:03.226317 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6fd8ce-78d3-47e5-81e3-858fe0e89c01 None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:01:03.323721 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6fd8ce-78d3-47e5-81e3-858fe0e89c01 None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:01:03.332829 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:01:03.357051 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6374e8-5ba7-468a-bf15-d3ffd7bc9e6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:01:03.357695 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6374e8-5ba7-468a-bf15-d3ffd7bc9e6d None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:01:03.357975 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6374e8-5ba7-468a-bf15-d3ffd7bc9e6d None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:01:03.358249 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6374e8-5ba7-468a-bf15-d3ffd7bc9e6d None admin] PATH_INFO: `/v3/domains` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:01:03.358879 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6374e8-5ba7-468a-bf15-d3ffd7bc9e6d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:01:03.359182 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6374e8-5ba7-468a-bf15-d3ffd7bc9e6d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:01:03.360577 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6374e8-5ba7-468a-bf15-d3ffd7bc9e6d None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:01:03.371977 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 618/1238] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Aug 24 06:01:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:01:03.377706 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e024ac-008f-4736-8380-0b933b6ec9bf None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:01:03.476237 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e024ac-008f-4736-8380-0b933b6ec9bf None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:01:03.509854 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88417eee-b8b0-49e7-81b5-35d01f146095 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:01:03.510474 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88417eee-b8b0-49e7-81b5-35d01f146095 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:01:03.510755 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88417eee-b8b0-49e7-81b5-35d01f146095 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:01:03.511030 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88417eee-b8b0-49e7-81b5-35d01f146095 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:01:03.514285 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:01:03.519422 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88417eee-b8b0-49e7-81b5-35d01f146095 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:01:03.520929 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88417eee-b8b0-49e7-81b5-35d01f146095 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:01:03.531733 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-88417eee-b8b0-49e7-81b5-35d01f146095 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 24 06:01:03.531733 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:01:03.531733 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:01:03.531733 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:01:03.531733 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:01:03.531733 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:01:03.531733 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:01:03.531733 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:01:03.531733 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:01:03.531733 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:01:03.531733 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:01:03.531733 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:01:03.531733 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 06:01:03.531733 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 06:01:03.531733 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 06:01:03.531733 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 06:01:03.532834 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:01:03.532834 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:01:03.532834 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 06:01:03.532834 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 06:01:03.532834 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 06:01:03.532834 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 06:01:03.532834 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:01:03.532834 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:01:03.532834 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:01:03.532834 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:01:03.532834 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:01:03.532834 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 24 06:01:03.532834 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:01:03.532834 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:01:03.532834 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:01:03.532834 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:01:03.532834 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:01:03.532834 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:01:03.534124 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:01:03.534124 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:01:03.534124 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 06:01:03.534124 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 06:01:03.534124 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 06:01:03.534124 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 06:01:03.534124 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 06:01:03.534124 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 06:01:03.534124 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 24 06:01:03.534124 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 24 06:01:03.534124 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 621/1239] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Aug 24 06:01:03 2022] GET /identity/v3/users/placement => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:01:03.538708 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6374e8-5ba7-468a-bf15-d3ffd7bc9e6d None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:01:03.638575 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6374e8-5ba7-468a-bf15-d3ffd7bc9e6d None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:01:03.671795 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264f7cd0-1770-4de5-b3ab-051a2ef1d53f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:01:03.672388 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264f7cd0-1770-4de5-b3ab-051a2ef1d53f None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:01:03.674078 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264f7cd0-1770-4de5-b3ab-051a2ef1d53f None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:01:03.674078 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264f7cd0-1770-4de5-b3ab-051a2ef1d53f None admin] PATH_INFO: `/v3/users` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:01:03.674078 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264f7cd0-1770-4de5-b3ab-051a2ef1d53f None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:01:03.674078 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264f7cd0-1770-4de5-b3ab-051a2ef1d53f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:01:03.675315 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264f7cd0-1770-4de5-b3ab-051a2ef1d53f None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:01:03.692274 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 619/1240] 10.222.0.62 () {56 vars in 1143 bytes} [Wed Aug 24 06:01:03 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:01:03.697897 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88417eee-b8b0-49e7-81b5-35d01f146095 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:01:03.796295 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88417eee-b8b0-49e7-81b5-35d01f146095 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:01:03.829802 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f276ae8b-cea1-476e-9c70-fd9f539ac341 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:01:03.830395 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f276ae8b-cea1-476e-9c70-fd9f539ac341 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:01:03.830669 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f276ae8b-cea1-476e-9c70-fd9f539ac341 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:01:03.830948 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f276ae8b-cea1-476e-9c70-fd9f539ac341 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:01:03.889392 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:01:03.895763 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f276ae8b-cea1-476e-9c70-fd9f539ac341 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:01:03.897235 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f276ae8b-cea1-476e-9c70-fd9f539ac341 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:01:03.994809 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 622/1241] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Aug 24 06:01:03 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 06:01:04.001256 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264f7cd0-1770-4de5-b3ab-051a2ef1d53f None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:01:04.027399 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:01:04.096690 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264f7cd0-1770-4de5-b3ab-051a2ef1d53f None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:01:04.130785 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05106c97-37dd-4ab7-84ac-a9ebbf623dc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:01:04.131393 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05106c97-37dd-4ab7-84ac-a9ebbf623dc3 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:01:04.131670 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05106c97-37dd-4ab7-84ac-a9ebbf623dc3 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:01:04.131966 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05106c97-37dd-4ab7-84ac-a9ebbf623dc3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:01:04.149848 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05106c97-37dd-4ab7-84ac-a9ebbf623dc3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:01:04.149848 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05106c97-37dd-4ab7-84ac-a9ebbf623dc3 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:01:04.157369 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application [None req-05106c97-37dd-4ab7-84ac-a9ebbf623dc3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:01:04.157369 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:01:04.157369 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:01:04.157369 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:01:04.157369 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:01:04.157369 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:01:04.157369 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:01:04.157369 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:01:04.157369 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:01:04.157369 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:01:04.157369 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:01:04.157369 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:01:04.157369 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 06:01:04.157369 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 06:01:04.157369 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 06:01:04.157369 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 06:01:04.158466 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:01:04.158466 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:01:04.158466 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:01:04.158466 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:01:04.158466 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:01:04.158466 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:01:04.158466 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:01:04.158466 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application with Lock( Aug 24 06:01:04.158466 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:01:04.158466 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:01:04.158466 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:01:04.158466 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:01:04.158466 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:01:04.158466 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:01:04.158466 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:01:04.158466 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:01:04.158466 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 06:01:04.159531 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 06:01:04.159531 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:01:04.159531 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Aug 24 06:01:04.159531 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 620/1242] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Aug 24 06:01:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:01:04.164206 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f276ae8b-cea1-476e-9c70-fd9f539ac341 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:01:04.266726 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:01:04.272354 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f276ae8b-cea1-476e-9c70-fd9f539ac341 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:01:04.305705 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c7a14b-db56-425a-8741-a77415172d7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:01:04.306297 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c7a14b-db56-425a-8741-a77415172d7f None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:01:04.306575 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c7a14b-db56-425a-8741-a77415172d7f None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:01:04.306842 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c7a14b-db56-425a-8741-a77415172d7f None admin] PATH_INFO: `/v3/domains` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:01:04.307463 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c7a14b-db56-425a-8741-a77415172d7f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:01:04.307759 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c7a14b-db56-425a-8741-a77415172d7f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:01:04.309155 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c7a14b-db56-425a-8741-a77415172d7f None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:01:04.319982 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 623/1243] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Aug 24 06:01:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:01:04.325872 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05106c97-37dd-4ab7-84ac-a9ebbf623dc3 None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:01:04.401471 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:01:04.422658 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05106c97-37dd-4ab7-84ac-a9ebbf623dc3 None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:01:04.453991 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b3c545-7004-42e6-8962-8ac56adbe5c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:01:04.454597 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b3c545-7004-42e6-8962-8ac56adbe5c2 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:01:04.454870 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b3c545-7004-42e6-8962-8ac56adbe5c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:01:04.455138 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b3c545-7004-42e6-8962-8ac56adbe5c2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:01:04.461461 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b3c545-7004-42e6-8962-8ac56adbe5c2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:01:04.462882 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b3c545-7004-42e6-8962-8ac56adbe5c2 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:01:04.474084 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application [None req-c8b3c545-7004-42e6-8962-8ac56adbe5c2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 06:01:04.474084 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:01:04.474084 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:01:04.474084 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:01:04.474084 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:01:04.474084 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:01:04.474084 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:01:04.474084 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:01:04.474084 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:01:04.474084 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:01:04.474084 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:01:04.474084 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:01:04.474084 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 06:01:04.474084 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 06:01:04.474084 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 06:01:04.476989 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 06:01:04.476989 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:01:04.476989 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:01:04.476989 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:01:04.476989 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:01:04.476989 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:01:04.476989 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:01:04.476989 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:01:04.476989 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application with Lock( Aug 24 06:01:04.476989 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:01:04.476989 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:01:04.476989 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:01:04.476989 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:01:04.476989 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:01:04.476989 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:01:04.476989 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:01:04.476989 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:01:04.478165 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 06:01:04.478165 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 06:01:04.478165 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 06:01:04.478165 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 06:01:04.478165 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 06:01:04.478165 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 06:01:04.478165 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 06:01:04.478165 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Aug 24 06:01:04.478165 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 621/1244] 10.222.0.62 () {56 vars in 1111 bytes} [Wed Aug 24 06:01:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:01:04.481593 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c7a14b-db56-425a-8741-a77415172d7f None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:01:04.579620 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c7a14b-db56-425a-8741-a77415172d7f None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:01:04.613522 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298606f8-8078-4f7a-8319-fb5e4c88fe94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:01:04.614121 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298606f8-8078-4f7a-8319-fb5e4c88fe94 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:01:04.614398 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298606f8-8078-4f7a-8319-fb5e4c88fe94 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:01:04.614670 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298606f8-8078-4f7a-8319-fb5e4c88fe94 None admin] PATH_INFO: `/v3/projects` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:01:04.615345 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298606f8-8078-4f7a-8319-fb5e4c88fe94 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:01:04.615648 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298606f8-8078-4f7a-8319-fb5e4c88fe94 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:01:04.617174 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298606f8-8078-4f7a-8319-fb5e4c88fe94 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:01:04.628791 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 624/1245] 10.222.0.62 () {56 vars in 1148 bytes} [Wed Aug 24 06:01:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:01:04.634869 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b3c545-7004-42e6-8962-8ac56adbe5c2 None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:01:04.736587 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b3c545-7004-42e6-8962-8ac56adbe5c2 None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:01:04.769353 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33f1de5-7391-4b87-af40-2ddc4d83a2b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:01:04.769977 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33f1de5-7391-4b87-af40-2ddc4d83a2b6 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:01:04.770250 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33f1de5-7391-4b87-af40-2ddc4d83a2b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:01:04.770516 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33f1de5-7391-4b87-af40-2ddc4d83a2b6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:01:04.771218 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33f1de5-7391-4b87-af40-2ddc4d83a2b6 None admin] RBAC: Adding query filter params (role.id=5d95a5fc0f324a92905533aa8d34ac15, scope.project.id=181f4762037e454489d9552cc603179c, user.id=abe9d8c9291845dfa4ef55034e5c9e32) {{(pid=81851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:01:04.771526 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33f1de5-7391-4b87-af40-2ddc4d83a2b6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:01:04.772963 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33f1de5-7391-4b87-af40-2ddc4d83a2b6 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:01:04.778607 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 622/1246] 10.222.0.62 () {56 vars in 1374 bytes} [Wed Aug 24 06:01:04 2022] GET /identity/v3/role_assignments?user.id=abe9d8c9291845dfa4ef55034e5c9e32&scope.project.id=181f4762037e454489d9552cc603179c&role.id=5d95a5fc0f324a92905533aa8d34ac15 => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:01:06.724999 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22501ff2-464e-4142-bdf3-291dd37d908f None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:01:06.725444 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22501ff2-464e-4142-bdf3-291dd37d908f None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:01:06.725554 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22501ff2-464e-4142-bdf3-291dd37d908f None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:01:06.726945 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 625/1247] 10.222.0.62 () {54 vars in 927 bytes} [Wed Aug 24 06:01:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:01:06.732013 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5e6e23-281c-4521-90f4-9899a6cb5e97 None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:01:06.732425 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5e6e23-281c-4521-90f4-9899a6cb5e97 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:01:06.732681 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5e6e23-281c-4521-90f4-9899a6cb5e97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:01:06.739375 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:01:06.751426 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-cc5e6e23-281c-4521-90f4-9899a6cb5e97 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:01:06.752620 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-cc5e6e23-281c-4521-90f4-9899a6cb5e97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:01:06.817326 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 623/1248] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Aug 24 06:01:06 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 06:01:07.081564 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22501ff2-464e-4142-bdf3-291dd37d908f None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:01:07.099991 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:01:07.181741 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22501ff2-464e-4142-bdf3-291dd37d908f None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:01:07.215292 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c79fb12-fae5-4da9-a118-a905f5f9c9ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:01:07.215905 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c79fb12-fae5-4da9-a118-a905f5f9c9ba None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:01:07.216179 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c79fb12-fae5-4da9-a118-a905f5f9c9ba None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:01:07.216474 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c79fb12-fae5-4da9-a118-a905f5f9c9ba None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:01:07.226361 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c79fb12-fae5-4da9-a118-a905f5f9c9ba None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 06:01:07.230975 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-5c79fb12-fae5-4da9-a118-a905f5f9c9ba None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 06:01:07.230975 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:01:07.230975 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:01:07.230975 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:01:07.230975 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:01:07.230975 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:01:07.230975 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:01:07.230975 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:01:07.230975 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:01:07.230975 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:01:07.230975 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:01:07.230975 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:01:07.230975 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 06:01:07.230975 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 06:01:07.230975 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 06:01:07.230975 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 06:01:07.232052 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 06:01:07.232052 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 06:01:07.232052 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 06:01:07.232052 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 06:01:07.232052 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:01:07.232052 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:01:07.232052 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:01:07.232052 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:01:07.232052 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:01:07.232052 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:01:07.232052 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:01:07.232052 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 24 06:01:07.232052 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:01:07.232052 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:01:07.232052 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:01:07.232052 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:01:07.237469 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:01:07.237469 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:01:07.237469 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:01:07.237469 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:01:07.237469 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 06:01:07.237469 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 06:01:07.237469 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 06:01:07.237469 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 06:01:07.237469 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 06:01:07.237469 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 06:01:07.237469 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 06:01:07.237469 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 24 06:01:07.237469 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 626/1249] 10.222.0.62 () {56 vars in 1096 bytes} [Wed Aug 24 06:01:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:01:07.238365 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5e6e23-281c-4521-90f4-9899a6cb5e97 None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:01:07.337246 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5e6e23-281c-4521-90f4-9899a6cb5e97 None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:01:07.365394 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:01:07.370121 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8961e9-fb9a-450f-9378-c050b549acc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:01:07.370720 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8961e9-fb9a-450f-9378-c050b549acc0 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:01:07.371032 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8961e9-fb9a-450f-9378-c050b549acc0 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:01:07.371306 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8961e9-fb9a-450f-9378-c050b549acc0 None admin] PATH_INFO: `/v3/roles` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:01:07.371960 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8961e9-fb9a-450f-9378-c050b549acc0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:01:07.372260 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8961e9-fb9a-450f-9378-c050b549acc0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:01:07.374222 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8961e9-fb9a-450f-9378-c050b549acc0 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:01:07.383441 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 624/1250] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Aug 24 06:01:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:01:07.389232 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c79fb12-fae5-4da9-a118-a905f5f9c9ba None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:01:07.463468 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:01:07.484702 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c79fb12-fae5-4da9-a118-a905f5f9c9ba None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:01:07.516959 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2b7cc9-79e3-419e-a6ef-f81aa3a5464f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:01:07.517568 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2b7cc9-79e3-419e-a6ef-f81aa3a5464f None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:01:07.523729 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2b7cc9-79e3-419e-a6ef-f81aa3a5464f None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:01:07.523729 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2b7cc9-79e3-419e-a6ef-f81aa3a5464f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:01:07.527394 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2b7cc9-79e3-419e-a6ef-f81aa3a5464f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:01:07.528890 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2b7cc9-79e3-419e-a6ef-f81aa3a5464f None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:01:07.542375 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-0e2b7cc9-79e3-419e-a6ef-f81aa3a5464f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:01:07.542375 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:01:07.542375 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:01:07.542375 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:01:07.542375 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:01:07.542375 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:01:07.542375 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:01:07.542375 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:01:07.542375 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:01:07.542375 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:01:07.542375 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:01:07.542375 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:01:07.542375 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 06:01:07.542375 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 06:01:07.542375 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 06:01:07.542375 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 06:01:07.543492 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:01:07.543492 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:01:07.543492 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:01:07.543492 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:01:07.543492 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:01:07.543492 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:01:07.543492 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:01:07.543492 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 24 06:01:07.543492 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:01:07.543492 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:01:07.543492 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:01:07.543492 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:01:07.543492 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:01:07.543492 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:01:07.543492 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:01:07.543492 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:01:07.543492 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 06:01:07.545086 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 06:01:07.545086 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:01:07.545086 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 24 06:01:07.545086 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 627/1251] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Aug 24 06:01:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:01:07.549534 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8961e9-fb9a-450f-9378-c050b549acc0 None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:01:07.647401 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8961e9-fb9a-450f-9378-c050b549acc0 None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:01:07.679767 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a056b2c-4db0-47d6-9967-1a23ca3b7b50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:01:07.684114 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a056b2c-4db0-47d6-9967-1a23ca3b7b50 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:01:07.684114 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a056b2c-4db0-47d6-9967-1a23ca3b7b50 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:01:07.684114 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a056b2c-4db0-47d6-9967-1a23ca3b7b50 None admin] PATH_INFO: `/v3/domains` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:01:07.684114 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a056b2c-4db0-47d6-9967-1a23ca3b7b50 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:01:07.684114 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a056b2c-4db0-47d6-9967-1a23ca3b7b50 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:01:07.684114 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a056b2c-4db0-47d6-9967-1a23ca3b7b50 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:01:07.695273 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 625/1252] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Aug 24 06:01:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 06:01:07.701021 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2b7cc9-79e3-419e-a6ef-f81aa3a5464f None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:01:07.800392 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2b7cc9-79e3-419e-a6ef-f81aa3a5464f None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:01:07.825819 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:01:07.833409 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345da714-9a65-492b-8935-f76c1e9dc5f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:01:07.834031 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345da714-9a65-492b-8935-f76c1e9dc5f9 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:01:07.834309 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345da714-9a65-492b-8935-f76c1e9dc5f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:01:07.834579 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345da714-9a65-492b-8935-f76c1e9dc5f9 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:01:07.841758 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345da714-9a65-492b-8935-f76c1e9dc5f9 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:01:07.843243 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345da714-9a65-492b-8935-f76c1e9dc5f9 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:01:07.854044 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-345da714-9a65-492b-8935-f76c1e9dc5f9 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 24 06:01:07.854044 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:01:07.854044 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:01:07.854044 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:01:07.854044 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:01:07.854044 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:01:07.854044 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:01:07.854044 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:01:07.854044 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:01:07.854044 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:01:07.854044 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:01:07.854044 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:01:07.854044 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 06:01:07.854044 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 06:01:07.854044 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 06:01:07.854044 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 06:01:07.855112 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:01:07.855112 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:01:07.855112 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 06:01:07.855112 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 06:01:07.855112 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 06:01:07.855112 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 06:01:07.855112 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:01:07.855112 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:01:07.855112 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:01:07.855112 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:01:07.855112 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:01:07.855112 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 24 06:01:07.855112 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:01:07.855112 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:01:07.855112 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:01:07.855112 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:01:07.855112 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:01:07.855112 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:01:07.856466 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:01:07.856466 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:01:07.856466 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 06:01:07.856466 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 06:01:07.856466 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 06:01:07.856466 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 06:01:07.856466 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 06:01:07.856466 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 06:01:07.856466 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 24 06:01:07.856466 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 24 06:01:07.856466 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 628/1253] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Aug 24 06:01:07 2022] GET /identity/v3/users/placement => generated 87 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:01:07.861622 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a056b2c-4db0-47d6-9967-1a23ca3b7b50 None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:01:07.916222 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:01:07.960535 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a056b2c-4db0-47d6-9967-1a23ca3b7b50 None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:01:07.993067 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67749114-fa74-42b1-8b90-be0e0cb8198a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:01:07.993655 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67749114-fa74-42b1-8b90-be0e0cb8198a None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:01:07.993929 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67749114-fa74-42b1-8b90-be0e0cb8198a None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:01:07.994195 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67749114-fa74-42b1-8b90-be0e0cb8198a None admin] PATH_INFO: `/v3/users` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:01:07.994903 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67749114-fa74-42b1-8b90-be0e0cb8198a None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:01:07.995202 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67749114-fa74-42b1-8b90-be0e0cb8198a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:01:07.996632 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67749114-fa74-42b1-8b90-be0e0cb8198a None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:01:08.013282 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 626/1254] 10.222.0.62 () {56 vars in 1143 bytes} [Wed Aug 24 06:01:07 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:01:08.019079 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345da714-9a65-492b-8935-f76c1e9dc5f9 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:01:08.115306 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345da714-9a65-492b-8935-f76c1e9dc5f9 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:01:08.146746 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e1a769-414b-4da3-a910-17e4d752517f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:01:08.147331 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e1a769-414b-4da3-a910-17e4d752517f None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:01:08.147605 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e1a769-414b-4da3-a910-17e4d752517f None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:01:08.147873 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e1a769-414b-4da3-a910-17e4d752517f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:01:08.201102 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:01:08.212960 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e1a769-414b-4da3-a910-17e4d752517f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:01:08.216152 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e1a769-414b-4da3-a910-17e4d752517f None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:01:08.310458 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 629/1255] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Aug 24 06:01:08 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 06:01:08.316297 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67749114-fa74-42b1-8b90-be0e0cb8198a None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:01:08.413319 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67749114-fa74-42b1-8b90-be0e0cb8198a None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:01:08.430839 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:01:08.451812 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8662f72d-d072-4d66-9375-fe9804df2286 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:01:08.452372 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8662f72d-d072-4d66-9375-fe9804df2286 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:01:08.452688 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8662f72d-d072-4d66-9375-fe9804df2286 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:01:08.452961 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8662f72d-d072-4d66-9375-fe9804df2286 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:01:08.462607 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8662f72d-d072-4d66-9375-fe9804df2286 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:01:08.464043 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8662f72d-d072-4d66-9375-fe9804df2286 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:01:08.478665 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application [None req-8662f72d-d072-4d66-9375-fe9804df2286 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:01:08.478665 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:01:08.478665 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:01:08.478665 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:01:08.478665 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:01:08.478665 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:01:08.478665 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:01:08.478665 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:01:08.478665 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:01:08.478665 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:01:08.478665 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:01:08.478665 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:01:08.478665 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 06:01:08.478665 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 06:01:08.478665 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 06:01:08.478665 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 06:01:08.479840 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:01:08.479840 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:01:08.479840 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:01:08.479840 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:01:08.479840 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:01:08.479840 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:01:08.479840 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:01:08.479840 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application with Lock( Aug 24 06:01:08.479840 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:01:08.479840 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:01:08.479840 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:01:08.479840 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:01:08.479840 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:01:08.479840 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:01:08.479840 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:01:08.479840 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:01:08.479840 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 06:01:08.480955 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 06:01:08.480955 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:01:08.480955 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Aug 24 06:01:08.480955 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 627/1256] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Aug 24 06:01:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:01:08.485755 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e1a769-414b-4da3-a910-17e4d752517f None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:01:08.569348 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:01:08.581144 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e1a769-414b-4da3-a910-17e4d752517f None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:01:08.613546 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0091e8-8d6d-4b19-a966-7e94ce834203 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:01:08.614140 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0091e8-8d6d-4b19-a966-7e94ce834203 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:01:08.614418 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0091e8-8d6d-4b19-a966-7e94ce834203 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:01:08.614689 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0091e8-8d6d-4b19-a966-7e94ce834203 None admin] PATH_INFO: `/v3/domains` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:01:08.615310 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0091e8-8d6d-4b19-a966-7e94ce834203 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:01:08.615610 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0091e8-8d6d-4b19-a966-7e94ce834203 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:01:08.616989 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0091e8-8d6d-4b19-a966-7e94ce834203 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:01:08.627691 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 630/1257] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Aug 24 06:01:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:01:08.633516 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8662f72d-d072-4d66-9375-fe9804df2286 None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:01:08.737857 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8662f72d-d072-4d66-9375-fe9804df2286 None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:01:08.770958 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994d14ee-f770-4958-9863-474c3c8d0232 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:01:08.771570 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994d14ee-f770-4958-9863-474c3c8d0232 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:01:08.771846 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994d14ee-f770-4958-9863-474c3c8d0232 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:01:08.772123 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994d14ee-f770-4958-9863-474c3c8d0232 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:01:08.778494 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994d14ee-f770-4958-9863-474c3c8d0232 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:01:08.780000 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994d14ee-f770-4958-9863-474c3c8d0232 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:01:08.791357 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application [None req-994d14ee-f770-4958-9863-474c3c8d0232 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 06:01:08.791357 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:01:08.791357 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:01:08.791357 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:01:08.791357 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:01:08.791357 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:01:08.791357 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:01:08.791357 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:01:08.791357 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:01:08.791357 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:01:08.791357 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:01:08.791357 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:01:08.791357 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 06:01:08.791357 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 06:01:08.791357 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 06:01:08.792477 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 06:01:08.792477 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:01:08.792477 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:01:08.792477 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:01:08.792477 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:01:08.792477 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:01:08.792477 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:01:08.792477 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:01:08.792477 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application with Lock( Aug 24 06:01:08.792477 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:01:08.792477 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:01:08.792477 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:01:08.792477 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:01:08.792477 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:01:08.792477 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:01:08.792477 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:01:08.792477 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:01:08.793599 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 06:01:08.793599 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 06:01:08.793599 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 06:01:08.793599 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 06:01:08.793599 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 06:01:08.793599 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 06:01:08.793599 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 06:01:08.793599 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Aug 24 06:01:08.793599 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 628/1258] 10.222.0.62 () {56 vars in 1111 bytes} [Wed Aug 24 06:01:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:01:08.798810 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0091e8-8d6d-4b19-a966-7e94ce834203 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:01:08.896129 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0091e8-8d6d-4b19-a966-7e94ce834203 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:01:08.927920 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbaadd9-dd55-44e1-b51f-b349f881304f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:01:08.928533 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbaadd9-dd55-44e1-b51f-b349f881304f None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:01:08.928815 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbaadd9-dd55-44e1-b51f-b349f881304f None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:01:08.929088 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbaadd9-dd55-44e1-b51f-b349f881304f None admin] PATH_INFO: `/v3/projects` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:01:08.929763 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbaadd9-dd55-44e1-b51f-b349f881304f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:01:08.930069 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbaadd9-dd55-44e1-b51f-b349f881304f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:01:08.931504 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbaadd9-dd55-44e1-b51f-b349f881304f None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:01:08.943990 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 631/1259] 10.222.0.62 () {56 vars in 1148 bytes} [Wed Aug 24 06:01:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:01:08.950073 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994d14ee-f770-4958-9863-474c3c8d0232 None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:01:08.975389 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:01:09.046896 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994d14ee-f770-4958-9863-474c3c8d0232 None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:01:09.080092 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6707775d-e47e-405b-bb57-fae227762777 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:01:09.084139 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6707775d-e47e-405b-bb57-fae227762777 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:01:09.084139 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6707775d-e47e-405b-bb57-fae227762777 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:01:09.084139 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6707775d-e47e-405b-bb57-fae227762777 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:01:09.084139 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6707775d-e47e-405b-bb57-fae227762777 None admin] RBAC: Adding query filter params (role.id=49e47cd157d743f6905f1db07ec878a7, scope.project.id=181f4762037e454489d9552cc603179c, user.id=abe9d8c9291845dfa4ef55034e5c9e32) {{(pid=81851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:01:09.084139 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6707775d-e47e-405b-bb57-fae227762777 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:01:09.084139 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6707775d-e47e-405b-bb57-fae227762777 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:01:09.088799 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 629/1260] 10.222.0.62 () {56 vars in 1374 bytes} [Wed Aug 24 06:01:08 2022] GET /identity/v3/role_assignments?user.id=abe9d8c9291845dfa4ef55034e5c9e32&scope.project.id=181f4762037e454489d9552cc603179c&role.id=49e47cd157d743f6905f1db07ec878a7 => generated 259 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:01:10.938875 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3569321d-05d6-434f-a02f-a2536b9fd750 None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:01:10.939446 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3569321d-05d6-434f-a02f-a2536b9fd750 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:01:10.939446 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3569321d-05d6-434f-a02f-a2536b9fd750 None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:01:10.940757 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 632/1261] 10.222.0.62 () {54 vars in 927 bytes} [Wed Aug 24 06:01:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:01:10.945819 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef9d4b5-8de6-452a-b9fd-75753f1f76ca None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:01:10.946272 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef9d4b5-8de6-452a-b9fd-75753f1f76ca None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:01:10.946394 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef9d4b5-8de6-452a-b9fd-75753f1f76ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:01:10.964083 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-2ef9d4b5-8de6-452a-b9fd-75753f1f76ca None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:01:10.965229 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-2ef9d4b5-8de6-452a-b9fd-75753f1f76ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:01:11.030449 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 630/1262] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Aug 24 06:01:10 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 06:01:11.294482 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3569321d-05d6-434f-a02f-a2536b9fd750 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:01:11.305734 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:01:11.393257 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3569321d-05d6-434f-a02f-a2536b9fd750 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:01:11.426123 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dead05e5-700d-49fa-8416-9c08e1334ad4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:01:11.426748 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dead05e5-700d-49fa-8416-9c08e1334ad4 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:01:11.427008 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dead05e5-700d-49fa-8416-9c08e1334ad4 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:01:11.427293 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dead05e5-700d-49fa-8416-9c08e1334ad4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:01:11.438388 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dead05e5-700d-49fa-8416-9c08e1334ad4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 06:01:11.443280 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-dead05e5-700d-49fa-8416-9c08e1334ad4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 06:01:11.443280 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:01:11.443280 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:01:11.443280 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:01:11.443280 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:01:11.443280 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:01:11.443280 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:01:11.443280 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:01:11.443280 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:01:11.443280 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:01:11.443280 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:01:11.443280 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:01:11.443280 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 06:01:11.443280 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 06:01:11.443280 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 06:01:11.443280 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 06:01:11.444397 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 06:01:11.444397 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 06:01:11.444397 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 06:01:11.444397 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 06:01:11.444397 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:01:11.444397 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:01:11.444397 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:01:11.444397 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:01:11.444397 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:01:11.444397 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:01:11.444397 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:01:11.444397 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 24 06:01:11.444397 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:01:11.444397 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:01:11.444397 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:01:11.444397 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:01:11.445458 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:01:11.445458 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:01:11.445458 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:01:11.445458 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:01:11.445458 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 06:01:11.445458 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 06:01:11.445458 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 06:01:11.445458 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 06:01:11.445458 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 06:01:11.445458 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 06:01:11.445458 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 06:01:11.445458 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 24 06:01:11.445458 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 633/1263] 10.222.0.62 () {56 vars in 1096 bytes} [Wed Aug 24 06:01:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:01:11.450390 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef9d4b5-8de6-452a-b9fd-75753f1f76ca None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:01:11.468054 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:01:11.545629 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef9d4b5-8de6-452a-b9fd-75753f1f76ca None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:01:11.578205 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1432149b-f259-4df1-b0a4-683c4931325f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:01:11.578791 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1432149b-f259-4df1-b0a4-683c4931325f None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:01:11.579067 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1432149b-f259-4df1-b0a4-683c4931325f None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:01:11.579335 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1432149b-f259-4df1-b0a4-683c4931325f None admin] PATH_INFO: `/v3/roles` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:01:11.579979 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1432149b-f259-4df1-b0a4-683c4931325f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:01:11.580278 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1432149b-f259-4df1-b0a4-683c4931325f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:01:11.581651 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1432149b-f259-4df1-b0a4-683c4931325f None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:01:11.589300 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 631/1264] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Aug 24 06:01:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:01:11.595194 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dead05e5-700d-49fa-8416-9c08e1334ad4 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:01:11.667591 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:01:11.699987 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dead05e5-700d-49fa-8416-9c08e1334ad4 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:01:11.733273 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e344d60-05b6-418d-bbae-ab38a888783c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:01:11.733894 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e344d60-05b6-418d-bbae-ab38a888783c None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:01:11.734168 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e344d60-05b6-418d-bbae-ab38a888783c None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:01:11.734440 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e344d60-05b6-418d-bbae-ab38a888783c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:01:11.743996 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e344d60-05b6-418d-bbae-ab38a888783c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:01:11.745468 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e344d60-05b6-418d-bbae-ab38a888783c None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:01:11.759187 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-3e344d60-05b6-418d-bbae-ab38a888783c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:01:11.759187 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:01:11.759187 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:01:11.759187 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:01:11.759187 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:01:11.759187 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:01:11.759187 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:01:11.759187 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:01:11.759187 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:01:11.759187 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:01:11.759187 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:01:11.759187 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:01:11.759187 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 06:01:11.759187 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 06:01:11.759187 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 06:01:11.759187 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 06:01:11.760262 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:01:11.760262 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:01:11.760262 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:01:11.760262 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:01:11.760262 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:01:11.760262 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:01:11.760262 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:01:11.760262 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 24 06:01:11.760262 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:01:11.760262 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:01:11.760262 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:01:11.760262 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:01:11.760262 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:01:11.760262 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:01:11.760262 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:01:11.760262 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:01:11.760262 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 06:01:11.761567 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 06:01:11.761567 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:01:11.761567 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 24 06:01:11.761567 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 634/1265] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Aug 24 06:01:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:01:11.766214 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1432149b-f259-4df1-b0a4-683c4931325f None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:01:11.847918 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:01:11.866996 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1432149b-f259-4df1-b0a4-683c4931325f None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:01:11.899268 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fab6755-5d9d-481a-86d2-043afac17fff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:01:11.899918 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fab6755-5d9d-481a-86d2-043afac17fff None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:01:11.900196 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fab6755-5d9d-481a-86d2-043afac17fff None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:01:11.900466 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fab6755-5d9d-481a-86d2-043afac17fff None admin] PATH_INFO: `/v3/domains` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:01:11.901128 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fab6755-5d9d-481a-86d2-043afac17fff None admin] RBAC: Adding query filter params (name=Default) {{(pid=81851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:01:11.901428 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fab6755-5d9d-481a-86d2-043afac17fff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:01:11.902782 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fab6755-5d9d-481a-86d2-043afac17fff None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:01:11.914947 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 632/1266] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Aug 24 06:01:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:01:11.921672 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e344d60-05b6-418d-bbae-ab38a888783c None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:01:12.018596 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e344d60-05b6-418d-bbae-ab38a888783c None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:01:12.038200 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:01:12.052448 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0759fda8-442d-4dc7-bcc8-a40020a28488 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:01:12.053098 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0759fda8-442d-4dc7-bcc8-a40020a28488 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:01:12.053381 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0759fda8-442d-4dc7-bcc8-a40020a28488 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:01:12.053657 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0759fda8-442d-4dc7-bcc8-a40020a28488 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:01:12.060868 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0759fda8-442d-4dc7-bcc8-a40020a28488 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:01:12.062306 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0759fda8-442d-4dc7-bcc8-a40020a28488 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:01:12.073121 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-0759fda8-442d-4dc7-bcc8-a40020a28488 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 24 06:01:12.073121 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:01:12.073121 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:01:12.073121 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:01:12.073121 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:01:12.073121 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:01:12.073121 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:01:12.073121 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:01:12.073121 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:01:12.073121 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:01:12.073121 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:01:12.073121 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:01:12.073121 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 06:01:12.073121 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 06:01:12.073121 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 06:01:12.073121 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 06:01:12.074223 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:01:12.074223 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:01:12.074223 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 06:01:12.074223 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 06:01:12.074223 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 06:01:12.074223 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 06:01:12.074223 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:01:12.074223 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:01:12.074223 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:01:12.074223 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:01:12.074223 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:01:12.074223 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 24 06:01:12.074223 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:01:12.074223 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:01:12.074223 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:01:12.074223 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:01:12.074223 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:01:12.074223 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:01:12.075372 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:01:12.075372 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:01:12.075372 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 06:01:12.075372 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 06:01:12.075372 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 06:01:12.075372 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 06:01:12.075372 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 06:01:12.075372 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 06:01:12.075372 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 24 06:01:12.075372 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 24 06:01:12.075372 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 635/1267] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Aug 24 06:01:11 2022] GET /identity/v3/users/placement => generated 87 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:01:12.080568 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fab6755-5d9d-481a-86d2-043afac17fff None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:01:12.178699 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fab6755-5d9d-481a-86d2-043afac17fff None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:01:12.210811 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd0378e-cd32-4c06-9fd2-ef9d4c13f9d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:01:12.211405 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd0378e-cd32-4c06-9fd2-ef9d4c13f9d5 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:01:12.211683 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd0378e-cd32-4c06-9fd2-ef9d4c13f9d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:01:12.211952 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd0378e-cd32-4c06-9fd2-ef9d4c13f9d5 None admin] PATH_INFO: `/v3/users` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:01:12.212698 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd0378e-cd32-4c06-9fd2-ef9d4c13f9d5 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:01:12.213000 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd0378e-cd32-4c06-9fd2-ef9d4c13f9d5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:01:12.214415 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd0378e-cd32-4c06-9fd2-ef9d4c13f9d5 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:01:12.223963 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:01:12.231839 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 633/1268] 10.222.0.62 () {56 vars in 1143 bytes} [Wed Aug 24 06:01:12 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:01:12.237527 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0759fda8-442d-4dc7-bcc8-a40020a28488 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:01:12.336173 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0759fda8-442d-4dc7-bcc8-a40020a28488 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:01:12.368729 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6a0c0d-50d8-431b-bb0e-696b26fd0418 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:01:12.369345 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6a0c0d-50d8-431b-bb0e-696b26fd0418 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:01:12.369627 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6a0c0d-50d8-431b-bb0e-696b26fd0418 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:01:12.369896 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6a0c0d-50d8-431b-bb0e-696b26fd0418 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:01:12.380746 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6a0c0d-50d8-431b-bb0e-696b26fd0418 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:01:12.382180 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6a0c0d-50d8-431b-bb0e-696b26fd0418 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:01:12.396315 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-0c6a0c0d-50d8-431b-bb0e-696b26fd0418 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:01:12.396315 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:01:12.396315 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:01:12.396315 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:01:12.396315 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:01:12.396315 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:01:12.396315 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:01:12.396315 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:01:12.396315 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:01:12.396315 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:01:12.396315 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:01:12.396315 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:01:12.396315 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 06:01:12.396315 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 06:01:12.396315 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 06:01:12.396315 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 06:01:12.397796 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:01:12.397796 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:01:12.397796 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:01:12.397796 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:01:12.397796 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:01:12.397796 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:01:12.397796 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:01:12.397796 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 24 06:01:12.397796 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:01:12.397796 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:01:12.397796 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:01:12.397796 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:01:12.397796 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:01:12.397796 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:01:12.397796 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:01:12.397796 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:01:12.397796 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 06:01:12.398716 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 06:01:12.398716 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:01:12.398716 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 24 06:01:12.398716 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 636/1269] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Aug 24 06:01:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:01:12.403411 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd0378e-cd32-4c06-9fd2-ef9d4c13f9d5 None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:01:12.501992 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd0378e-cd32-4c06-9fd2-ef9d4c13f9d5 None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:01:12.534677 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd67b806-ea4d-4ee4-ad00-f6c2216a10d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:01:12.535268 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd67b806-ea4d-4ee4-ad00-f6c2216a10d1 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:01:12.535645 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd67b806-ea4d-4ee4-ad00-f6c2216a10d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:01:12.535912 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd67b806-ea4d-4ee4-ad00-f6c2216a10d1 None admin] PATH_INFO: `/v3/domains` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:01:12.536568 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd67b806-ea4d-4ee4-ad00-f6c2216a10d1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:01:12.536877 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd67b806-ea4d-4ee4-ad00-f6c2216a10d1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:01:12.538216 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd67b806-ea4d-4ee4-ad00-f6c2216a10d1 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:01:12.549248 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 634/1270] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Aug 24 06:01:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:01:12.554902 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6a0c0d-50d8-431b-bb0e-696b26fd0418 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:01:12.590782 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:01:12.662693 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6a0c0d-50d8-431b-bb0e-696b26fd0418 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:01:12.695120 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44d9368-bbc4-41f5-8137-79dec9d7e435 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:01:12.695742 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44d9368-bbc4-41f5-8137-79dec9d7e435 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:01:12.696017 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44d9368-bbc4-41f5-8137-79dec9d7e435 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:01:12.696287 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44d9368-bbc4-41f5-8137-79dec9d7e435 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:01:12.703602 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44d9368-bbc4-41f5-8137-79dec9d7e435 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:01:12.705648 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44d9368-bbc4-41f5-8137-79dec9d7e435 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:01:12.716719 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-d44d9368-bbc4-41f5-8137-79dec9d7e435 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 06:01:12.716719 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:01:12.716719 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:01:12.716719 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:01:12.716719 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:01:12.716719 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:01:12.716719 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:01:12.716719 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:01:12.716719 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:01:12.716719 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:01:12.716719 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:01:12.716719 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:01:12.716719 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 06:01:12.716719 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 06:01:12.716719 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 06:01:12.717829 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 06:01:12.717829 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:01:12.717829 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:01:12.717829 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:01:12.717829 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:01:12.717829 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:01:12.717829 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:01:12.717829 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:01:12.717829 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 24 06:01:12.717829 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:01:12.717829 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:01:12.717829 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:01:12.717829 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:01:12.717829 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:01:12.717829 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:01:12.717829 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:01:12.717829 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:01:12.719097 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 06:01:12.719097 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 06:01:12.719097 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 06:01:12.719097 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 06:01:12.719097 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 06:01:12.719097 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 06:01:12.719097 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 06:01:12.719097 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 24 06:01:12.719097 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 637/1271] 10.222.0.62 () {56 vars in 1111 bytes} [Wed Aug 24 06:01:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:01:12.723699 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd67b806-ea4d-4ee4-ad00-f6c2216a10d1 None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:01:12.781041 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:01:12.821911 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd67b806-ea4d-4ee4-ad00-f6c2216a10d1 None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:01:12.853362 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14aa632e-98ef-4614-b3fb-d067b2039095 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:01:12.853964 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14aa632e-98ef-4614-b3fb-d067b2039095 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:01:12.854239 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14aa632e-98ef-4614-b3fb-d067b2039095 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:01:12.854506 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14aa632e-98ef-4614-b3fb-d067b2039095 None admin] PATH_INFO: `/v3/projects` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:01:12.855183 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14aa632e-98ef-4614-b3fb-d067b2039095 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:01:12.855483 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14aa632e-98ef-4614-b3fb-d067b2039095 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:01:12.856927 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14aa632e-98ef-4614-b3fb-d067b2039095 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:01:12.869362 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 635/1272] 10.222.0.62 () {56 vars in 1148 bytes} [Wed Aug 24 06:01:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:01:12.876004 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44d9368-bbc4-41f5-8137-79dec9d7e435 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:01:12.964636 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:01:12.972310 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44d9368-bbc4-41f5-8137-79dec9d7e435 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:01:13.005704 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88683ac8-cdc4-41e1-b9e8-7174c1dffe6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:01:13.006327 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88683ac8-cdc4-41e1-b9e8-7174c1dffe6a None admin] REQUEST_METHOD: `PUT` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:01:13.006608 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88683ac8-cdc4-41e1-b9e8-7174c1dffe6a None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:01:13.006880 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88683ac8-cdc4-41e1-b9e8-7174c1dffe6a None admin] PATH_INFO: `/v3/projects/181f4762037e454489d9552cc603179c/users/abe9d8c9291845dfa4ef55034e5c9e32/roles/49e47cd157d743f6905f1db07ec878a7` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:01:13.008776 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88683ac8-cdc4-41e1-b9e8-7174c1dffe6a None admin] RBAC: Authorizing `identity:create_grant(project_id=181f4762037e454489d9552cc603179c, user_id=abe9d8c9291845dfa4ef55034e5c9e32, role_id=49e47cd157d743f6905f1db07ec878a7)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:01:13.010245 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88683ac8-cdc4-41e1-b9e8-7174c1dffe6a None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:01:13.106668 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 638/1273] 10.222.0.62 () {58 vars in 1439 bytes} [Wed Aug 24 06:01:12 2022] PUT /identity/v3/projects/181f4762037e454489d9552cc603179c/users/abe9d8c9291845dfa4ef55034e5c9e32/roles/49e47cd157d743f6905f1db07ec878a7 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:01:14.973449 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b176db-92ca-4058-92ac-945e216863ec None None] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:01:14.973962 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b176db-92ca-4058-92ac-945e216863ec None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:01:14.973962 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b176db-92ca-4058-92ac-945e216863ec None None] PATH_INFO: `/` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:01:14.975381 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 636/1274] 10.222.0.62 () {54 vars in 927 bytes} [Wed Aug 24 06:01:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:01:14.980532 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58367264-48c3-4fb2-a97e-d57e1262a606 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:01:14.980960 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58367264-48c3-4fb2-a97e-d57e1262a606 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:01:14.981093 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58367264-48c3-4fb2-a97e-d57e1262a606 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:01:14.999378 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-58367264-48c3-4fb2-a97e-d57e1262a606 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:01:15.000586 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-58367264-48c3-4fb2-a97e-d57e1262a606 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:01:15.065750 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 639/1275] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Aug 24 06:01:14 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 06:01:15.332314 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b176db-92ca-4058-92ac-945e216863ec None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:01:15.432445 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b176db-92ca-4058-92ac-945e216863ec None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:01:15.449580 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:01:15.466207 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b35881-f366-4860-90b2-5b0cfdf7781e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:01:15.466844 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b35881-f366-4860-90b2-5b0cfdf7781e None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:01:15.467125 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b35881-f366-4860-90b2-5b0cfdf7781e None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:01:15.467393 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b35881-f366-4860-90b2-5b0cfdf7781e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:01:15.477188 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b35881-f366-4860-90b2-5b0cfdf7781e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 06:01:15.481909 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application [None req-28b35881-f366-4860-90b2-5b0cfdf7781e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 06:01:15.481909 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:01:15.481909 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:01:15.481909 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:01:15.481909 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:01:15.481909 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:01:15.481909 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:01:15.481909 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:01:15.481909 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:01:15.481909 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:01:15.481909 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:01:15.481909 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:01:15.481909 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 06:01:15.481909 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 06:01:15.481909 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 06:01:15.481909 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 06:01:15.483002 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 06:01:15.483002 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 06:01:15.483002 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 06:01:15.483002 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 06:01:15.483002 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:01:15.483002 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:01:15.483002 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:01:15.483002 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:01:15.483002 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:01:15.483002 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:01:15.483002 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:01:15.483002 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application with Lock( Aug 24 06:01:15.483002 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:01:15.483002 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:01:15.483002 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:01:15.483002 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:01:15.483873 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:01:15.483873 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:01:15.483873 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:01:15.483873 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:01:15.483873 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 06:01:15.483873 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 06:01:15.483873 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 06:01:15.483873 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 06:01:15.483873 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 06:01:15.483873 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 06:01:15.483873 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 06:01:15.483873 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Aug 24 06:01:15.483873 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 637/1276] 10.222.0.62 () {56 vars in 1096 bytes} [Wed Aug 24 06:01:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:01:15.488952 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58367264-48c3-4fb2-a97e-d57e1262a606 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:01:15.566911 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:01:15.584533 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58367264-48c3-4fb2-a97e-d57e1262a606 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:01:15.616253 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47760618-3959-43ec-9c6d-4ed849a43b0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:01:15.616877 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47760618-3959-43ec-9c6d-4ed849a43b0e None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:01:15.617153 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47760618-3959-43ec-9c6d-4ed849a43b0e None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:01:15.617421 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47760618-3959-43ec-9c6d-4ed849a43b0e None admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:01:15.618064 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47760618-3959-43ec-9c6d-4ed849a43b0e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:01:15.618364 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47760618-3959-43ec-9c6d-4ed849a43b0e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:01:15.619790 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47760618-3959-43ec-9c6d-4ed849a43b0e None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:01:15.627956 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 640/1277] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Aug 24 06:01:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:01:15.633704 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b35881-f366-4860-90b2-5b0cfdf7781e None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:01:15.733658 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b35881-f366-4860-90b2-5b0cfdf7781e None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:01:15.767991 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9d530e-969d-472b-ae90-f1ef14d8af29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:01:15.768622 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9d530e-969d-472b-ae90-f1ef14d8af29 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:01:15.768899 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9d530e-969d-472b-ae90-f1ef14d8af29 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:01:15.769166 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9d530e-969d-472b-ae90-f1ef14d8af29 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:01:15.778847 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9d530e-969d-472b-ae90-f1ef14d8af29 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:01:15.780302 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9d530e-969d-472b-ae90-f1ef14d8af29 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:01:15.794508 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application [None req-5b9d530e-969d-472b-ae90-f1ef14d8af29 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:01:15.794508 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:01:15.794508 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:01:15.794508 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:01:15.794508 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:01:15.794508 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:01:15.794508 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:01:15.794508 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:01:15.794508 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:01:15.794508 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:01:15.794508 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:01:15.794508 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:01:15.794508 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 06:01:15.794508 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 06:01:15.794508 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 06:01:15.794508 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 06:01:15.796116 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:01:15.796116 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:01:15.796116 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:01:15.796116 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:01:15.796116 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:01:15.796116 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:01:15.796116 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:01:15.796116 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application with Lock( Aug 24 06:01:15.796116 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:01:15.796116 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:01:15.796116 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:01:15.796116 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:01:15.796116 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:01:15.796116 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:01:15.796116 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:01:15.796116 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:01:15.796116 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 06:01:15.797162 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 06:01:15.797162 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:01:15.797162 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Aug 24 06:01:15.797162 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 638/1278] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Aug 24 06:01:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:01:15.802046 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47760618-3959-43ec-9c6d-4ed849a43b0e None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:01:15.899489 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47760618-3959-43ec-9c6d-4ed849a43b0e None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:01:15.932460 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f4503e-dbd7-43f9-87f3-d2fb64e9a077 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:01:15.933091 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f4503e-dbd7-43f9-87f3-d2fb64e9a077 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:01:15.933370 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f4503e-dbd7-43f9-87f3-d2fb64e9a077 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:01:15.933639 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f4503e-dbd7-43f9-87f3-d2fb64e9a077 None admin] PATH_INFO: `/v3/domains` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:01:15.934272 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f4503e-dbd7-43f9-87f3-d2fb64e9a077 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:01:15.934569 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f4503e-dbd7-43f9-87f3-d2fb64e9a077 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:01:15.936518 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f4503e-dbd7-43f9-87f3-d2fb64e9a077 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:01:15.948080 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 641/1279] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Aug 24 06:01:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:01:15.953899 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9d530e-969d-472b-ae90-f1ef14d8af29 None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:01:15.984835 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:01:16.052347 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9d530e-969d-472b-ae90-f1ef14d8af29 None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:01:16.085512 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b75d587-22d3-4f31-be51-83a0f6d9917f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:01:16.086184 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b75d587-22d3-4f31-be51-83a0f6d9917f None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:01:16.086466 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b75d587-22d3-4f31-be51-83a0f6d9917f None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:01:16.086739 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b75d587-22d3-4f31-be51-83a0f6d9917f None admin] PATH_INFO: `/v3/users/placement` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:01:16.093503 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b75d587-22d3-4f31-be51-83a0f6d9917f None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:01:16.094933 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b75d587-22d3-4f31-be51-83a0f6d9917f None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:01:16.106884 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application [None req-5b75d587-22d3-4f31-be51-83a0f6d9917f None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 24 06:01:16.106884 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:01:16.106884 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:01:16.106884 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:01:16.106884 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:01:16.106884 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:01:16.106884 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:01:16.106884 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:01:16.106884 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:01:16.106884 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:01:16.106884 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:01:16.106884 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:01:16.106884 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 06:01:16.106884 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 06:01:16.106884 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 06:01:16.106884 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 06:01:16.109684 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:01:16.109684 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:01:16.109684 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 06:01:16.109684 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 06:01:16.109684 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 06:01:16.109684 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 06:01:16.109684 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:01:16.109684 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:01:16.109684 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:01:16.109684 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:01:16.109684 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:01:16.109684 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application with Lock( Aug 24 06:01:16.109684 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:01:16.109684 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:01:16.109684 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:01:16.109684 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:01:16.109684 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:01:16.109684 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:01:16.110748 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:01:16.110748 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:01:16.110748 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 06:01:16.110748 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 06:01:16.110748 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 06:01:16.110748 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 06:01:16.110748 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 06:01:16.110748 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 06:01:16.110748 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 24 06:01:16.110748 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Aug 24 06:01:16.110748 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 639/1280] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Aug 24 06:01:15 2022] GET /identity/v3/users/placement => generated 87 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:01:16.114354 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f4503e-dbd7-43f9-87f3-d2fb64e9a077 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:01:16.119032 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:01:16.214626 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f4503e-dbd7-43f9-87f3-d2fb64e9a077 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:01:16.247729 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09dbfc44-cd94-43a7-a95a-30a477043729 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:01:16.248372 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dbfc44-cd94-43a7-a95a-30a477043729 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:01:16.248684 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dbfc44-cd94-43a7-a95a-30a477043729 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:01:16.248954 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dbfc44-cd94-43a7-a95a-30a477043729 None admin] PATH_INFO: `/v3/users` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:01:16.249669 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09dbfc44-cd94-43a7-a95a-30a477043729 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:01:16.249972 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09dbfc44-cd94-43a7-a95a-30a477043729 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:01:16.251382 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09dbfc44-cd94-43a7-a95a-30a477043729 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:01:16.268966 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 642/1281] 10.222.0.62 () {56 vars in 1143 bytes} [Wed Aug 24 06:01:16 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:01:16.274780 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b75d587-22d3-4f31-be51-83a0f6d9917f None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:01:16.366232 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:01:16.375589 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b75d587-22d3-4f31-be51-83a0f6d9917f None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:01:16.408096 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fe3414-af26-47cb-aba9-2f60cc3d72b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:01:16.408716 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fe3414-af26-47cb-aba9-2f60cc3d72b1 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:01:16.408993 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fe3414-af26-47cb-aba9-2f60cc3d72b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:01:16.409260 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fe3414-af26-47cb-aba9-2f60cc3d72b1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:01:16.473414 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76fe3414-af26-47cb-aba9-2f60cc3d72b1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:01:16.474794 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76fe3414-af26-47cb-aba9-2f60cc3d72b1 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:01:16.570827 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 640/1282] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Aug 24 06:01:16 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 06:01:16.577370 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09dbfc44-cd94-43a7-a95a-30a477043729 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:01:16.594174 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:01:16.636608 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:01:16.673747 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09dbfc44-cd94-43a7-a95a-30a477043729 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:01:16.706507 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4aa3732-88b9-4277-a7a0-c67358f812de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:01:16.707024 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4aa3732-88b9-4277-a7a0-c67358f812de None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:01:16.707303 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4aa3732-88b9-4277-a7a0-c67358f812de None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:01:16.707574 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4aa3732-88b9-4277-a7a0-c67358f812de None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:01:16.717903 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4aa3732-88b9-4277-a7a0-c67358f812de None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:01:16.719334 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4aa3732-88b9-4277-a7a0-c67358f812de None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:01:16.733160 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-f4aa3732-88b9-4277-a7a0-c67358f812de None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:01:16.733160 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:01:16.733160 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:01:16.733160 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:01:16.733160 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:01:16.733160 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:01:16.733160 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:01:16.733160 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:01:16.733160 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:01:16.733160 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:01:16.733160 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:01:16.733160 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:01:16.733160 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 06:01:16.733160 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 06:01:16.733160 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 06:01:16.733160 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 06:01:16.734264 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:01:16.734264 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:01:16.734264 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:01:16.734264 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:01:16.734264 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:01:16.734264 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:01:16.734264 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:01:16.734264 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 24 06:01:16.734264 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:01:16.734264 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:01:16.734264 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:01:16.734264 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:01:16.734264 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:01:16.734264 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:01:16.734264 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:01:16.734264 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:01:16.734264 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 06:01:16.735323 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 06:01:16.735323 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:01:16.735323 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 24 06:01:16.735323 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 643/1283] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Aug 24 06:01:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:01:16.739930 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fe3414-af26-47cb-aba9-2f60cc3d72b1 None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:01:16.839556 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fe3414-af26-47cb-aba9-2f60cc3d72b1 None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:01:16.872435 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75735a6d-01a0-40da-a442-74a3059dc69c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:01:16.873134 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75735a6d-01a0-40da-a442-74a3059dc69c None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:01:16.873412 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75735a6d-01a0-40da-a442-74a3059dc69c None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:01:16.873684 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75735a6d-01a0-40da-a442-74a3059dc69c None admin] PATH_INFO: `/v3/domains` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:01:16.874311 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75735a6d-01a0-40da-a442-74a3059dc69c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:01:16.874611 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75735a6d-01a0-40da-a442-74a3059dc69c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:01:16.875988 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75735a6d-01a0-40da-a442-74a3059dc69c None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:01:16.886765 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 641/1284] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Aug 24 06:01:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:01:16.893336 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4aa3732-88b9-4277-a7a0-c67358f812de None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:01:16.998222 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4aa3732-88b9-4277-a7a0-c67358f812de None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:01:17.011840 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:01:17.032398 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5672e095-03e0-4c19-801d-b1077d529d40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:01:17.033037 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5672e095-03e0-4c19-801d-b1077d529d40 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:01:17.033311 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5672e095-03e0-4c19-801d-b1077d529d40 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:01:17.033578 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5672e095-03e0-4c19-801d-b1077d529d40 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:01:17.040116 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5672e095-03e0-4c19-801d-b1077d529d40 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:01:17.041887 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5672e095-03e0-4c19-801d-b1077d529d40 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:01:17.053080 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-5672e095-03e0-4c19-801d-b1077d529d40 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 06:01:17.053080 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:01:17.053080 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:01:17.053080 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:01:17.053080 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:01:17.053080 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:01:17.053080 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:01:17.053080 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:01:17.053080 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:01:17.053080 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:01:17.053080 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:01:17.053080 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:01:17.053080 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 06:01:17.053080 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 06:01:17.053080 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 06:01:17.054137 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 06:01:17.054137 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:01:17.054137 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:01:17.054137 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:01:17.054137 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:01:17.054137 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:01:17.054137 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:01:17.054137 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:01:17.054137 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 24 06:01:17.054137 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:01:17.054137 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:01:17.054137 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:01:17.054137 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:01:17.054137 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:01:17.054137 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:01:17.054137 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:01:17.054137 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:01:17.055058 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 06:01:17.055058 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 06:01:17.055058 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 06:01:17.055058 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 06:01:17.055058 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 06:01:17.055058 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 06:01:17.055058 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 06:01:17.055058 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 24 06:01:17.055058 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 644/1285] 10.222.0.62 () {56 vars in 1111 bytes} [Wed Aug 24 06:01:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:01:17.060211 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75735a6d-01a0-40da-a442-74a3059dc69c None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:01:17.109876 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:01:17.158289 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75735a6d-01a0-40da-a442-74a3059dc69c None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:01:17.190975 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4e552f-3bc6-493d-b1bf-0cf06d5e9d12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:01:17.191562 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4e552f-3bc6-493d-b1bf-0cf06d5e9d12 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:01:17.191836 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4e552f-3bc6-493d-b1bf-0cf06d5e9d12 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:01:17.192104 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4e552f-3bc6-493d-b1bf-0cf06d5e9d12 None admin] PATH_INFO: `/v3/projects` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:01:17.192806 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4e552f-3bc6-493d-b1bf-0cf06d5e9d12 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:01:17.193106 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4e552f-3bc6-493d-b1bf-0cf06d5e9d12 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:01:17.194521 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4e552f-3bc6-493d-b1bf-0cf06d5e9d12 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:01:17.205948 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 642/1286] 10.222.0.62 () {56 vars in 1148 bytes} [Wed Aug 24 06:01:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:01:17.211944 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5672e095-03e0-4c19-801d-b1077d529d40 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:01:17.311249 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5672e095-03e0-4c19-801d-b1077d529d40 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:01:17.344789 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f95bbe-521c-46ce-8b7e-0b235cc75b10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:01:17.345378 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f95bbe-521c-46ce-8b7e-0b235cc75b10 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:01:17.345654 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f95bbe-521c-46ce-8b7e-0b235cc75b10 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:01:17.345927 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f95bbe-521c-46ce-8b7e-0b235cc75b10 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:01:17.346634 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f95bbe-521c-46ce-8b7e-0b235cc75b10 None admin] RBAC: Adding query filter params (role.id=49e47cd157d743f6905f1db07ec878a7, scope.project.id=181f4762037e454489d9552cc603179c, user.id=abe9d8c9291845dfa4ef55034e5c9e32) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:01:17.346940 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f95bbe-521c-46ce-8b7e-0b235cc75b10 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:01:17.348361 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f95bbe-521c-46ce-8b7e-0b235cc75b10 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:01:17.353998 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 645/1287] 10.222.0.62 () {56 vars in 1374 bytes} [Wed Aug 24 06:01:17 2022] GET /identity/v3/role_assignments?user.id=abe9d8c9291845dfa4ef55034e5c9e32&scope.project.id=181f4762037e454489d9552cc603179c&role.id=49e47cd157d743f6905f1db07ec878a7 => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:01:19.287809 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1733ce81-8475-4782-b919-6a3001f8cb93 None None] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:01:19.288348 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1733ce81-8475-4782-b919-6a3001f8cb93 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:01:19.288348 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1733ce81-8475-4782-b919-6a3001f8cb93 None None] PATH_INFO: `/` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:01:19.289709 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 643/1288] 10.222.0.62 () {54 vars in 927 bytes} [Wed Aug 24 06:01:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:01:19.295171 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6946c5f0-4887-42cc-be11-36a5e4f15c0c None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:01:19.295646 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6946c5f0-4887-42cc-be11-36a5e4f15c0c None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:01:19.296370 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6946c5f0-4887-42cc-be11-36a5e4f15c0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:01:19.314256 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-6946c5f0-4887-42cc-be11-36a5e4f15c0c None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:01:19.315454 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-6946c5f0-4887-42cc-be11-36a5e4f15c0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:01:19.345224 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:01:19.381202 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 646/1289] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Aug 24 06:01:19 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 06:01:19.661217 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1733ce81-8475-4782-b919-6a3001f8cb93 None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:01:19.762594 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1733ce81-8475-4782-b919-6a3001f8cb93 None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:01:19.773231 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:01:19.796266 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719359e0-388e-4e49-a292-62d9ef2b5d9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:01:19.796924 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719359e0-388e-4e49-a292-62d9ef2b5d9a None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:01:19.797209 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719359e0-388e-4e49-a292-62d9ef2b5d9a None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:01:19.797485 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719359e0-388e-4e49-a292-62d9ef2b5d9a None admin] PATH_INFO: `/v3/services/placement` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:01:19.798057 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719359e0-388e-4e49-a292-62d9ef2b5d9a None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:01:19.799500 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719359e0-388e-4e49-a292-62d9ef2b5d9a None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:01:19.808042 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application [None req-719359e0-388e-4e49-a292-62d9ef2b5d9a None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 06:01:19.808042 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:01:19.808042 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 06:01:19.808042 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 06:01:19.808042 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 06:01:19.808042 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 06:01:19.808042 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 06:01:19.808042 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 06:01:19.808042 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 06:01:19.808042 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Aug 24 06:01:19.808042 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 06:01:19.808042 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Aug 24 06:01:19.808042 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:01:19.808042 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:01:19.808042 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:01:19.808042 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:01:19.808042 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:01:19.809255 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:01:19.809255 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:01:19.809255 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:01:19.809255 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:01:19.809255 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:01:19.809255 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:01:19.809255 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 06:01:19.809255 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 06:01:19.809255 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 06:01:19.809255 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 06:01:19.809255 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:01:19.809255 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:01:19.809255 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:01:19.809255 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:01:19.809255 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:01:19.809255 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:01:19.810467 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:01:19.810467 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application with Lock( Aug 24 06:01:19.810467 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:01:19.810467 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:01:19.810467 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:01:19.810467 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:01:19.810467 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:01:19.810467 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:01:19.810467 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:01:19.810467 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:01:19.810467 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 06:01:19.810467 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 06:01:19.810467 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 06:01:19.810467 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Aug 24 06:01:19.810467 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 644/1290] 10.222.0.62 () {56 vars in 1117 bytes} [Wed Aug 24 06:01:19 2022] GET /identity/v3/services/placement => generated 90 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:01:19.815745 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6946c5f0-4887-42cc-be11-36a5e4f15c0c None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:01:19.923453 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6946c5f0-4887-42cc-be11-36a5e4f15c0c None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:01:19.955504 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a961856-0caf-430b-bd4b-af9981d1656b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:01:19.956109 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a961856-0caf-430b-bd4b-af9981d1656b None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:01:19.956433 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a961856-0caf-430b-bd4b-af9981d1656b None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:01:19.956741 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a961856-0caf-430b-bd4b-af9981d1656b None admin] PATH_INFO: `/v3/services` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:01:19.957362 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a961856-0caf-430b-bd4b-af9981d1656b None admin] RBAC: Adding query filter params (name=placement) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:01:19.957664 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a961856-0caf-430b-bd4b-af9981d1656b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:01:19.959013 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a961856-0caf-430b-bd4b-af9981d1656b None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:01:19.963928 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 647/1291] 10.222.0.62 () {56 vars in 1116 bytes} [Wed Aug 24 06:01:19 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:01:19.970390 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719359e0-388e-4e49-a292-62d9ef2b5d9a None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:01:20.070940 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719359e0-388e-4e49-a292-62d9ef2b5d9a None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:01:20.103512 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab20f180-ade6-4357-bf49-81d020346df5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:01:20.105722 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab20f180-ade6-4357-bf49-81d020346df5 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:01:20.105722 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab20f180-ade6-4357-bf49-81d020346df5 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:01:20.105722 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab20f180-ade6-4357-bf49-81d020346df5 None admin] PATH_INFO: `/v3/services` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:01:20.105722 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab20f180-ade6-4357-bf49-81d020346df5 None admin] RBAC: Adding query filter params (type=placement) {{(pid=81851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:01:20.105722 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab20f180-ade6-4357-bf49-81d020346df5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:01:20.106919 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab20f180-ade6-4357-bf49-81d020346df5 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:01:20.111829 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 645/1292] 10.222.0.62 () {56 vars in 1116 bytes} [Wed Aug 24 06:01:19 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:01:21.984067 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9601b2f4-406f-4d72-9403-050ed9c6b0dc None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:01:21.984660 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9601b2f4-406f-4d72-9403-050ed9c6b0dc None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:01:21.984660 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9601b2f4-406f-4d72-9403-050ed9c6b0dc None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:01:21.985961 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 648/1293] 10.222.0.62 () {54 vars in 927 bytes} [Wed Aug 24 06:01:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:01:21.991658 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c33e9a-af3e-4e5c-9687-92de561273fc None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:01:21.992117 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c33e9a-af3e-4e5c-9687-92de561273fc None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:01:21.992845 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c33e9a-af3e-4e5c-9687-92de561273fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:01:22.011661 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-79c33e9a-af3e-4e5c-9687-92de561273fc None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:01:22.012876 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-79c33e9a-af3e-4e5c-9687-92de561273fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:01:22.041614 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:01:22.078661 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 646/1294] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Aug 24 06:01:21 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 06:01:22.345100 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9601b2f4-406f-4d72-9403-050ed9c6b0dc None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:01:22.375502 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:01:22.443821 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9601b2f4-406f-4d72-9403-050ed9c6b0dc None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:01:22.476600 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc26797-326e-49ac-872f-4d0beb9390f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:01:22.477238 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc26797-326e-49ac-872f-4d0beb9390f1 None admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:01:22.477526 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc26797-326e-49ac-872f-4d0beb9390f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:01:22.477799 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc26797-326e-49ac-872f-4d0beb9390f1 None admin] PATH_INFO: `/v3/services` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:01:22.478440 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc26797-326e-49ac-872f-4d0beb9390f1 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:01:22.479801 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc26797-326e-49ac-872f-4d0beb9390f1 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:01:22.587127 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 649/1295] 10.222.0.62 () {60 vars in 1141 bytes} [Wed Aug 24 06:01:22 2022] POST /identity/v3/services => generated 246 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:01:24.471989 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b643a54-cf5a-4e3e-b3d6-8e3f552922d8 None None] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:01:24.511073 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b643a54-cf5a-4e3e-b3d6-8e3f552922d8 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:01:24.511073 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b643a54-cf5a-4e3e-b3d6-8e3f552922d8 None None] PATH_INFO: `/` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:01:24.511073 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 647/1296] 10.222.0.62 () {54 vars in 927 bytes} [Wed Aug 24 06:01:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:01:24.511440 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e18789-ef3f-4de2-a791-bf6ddcb1cf7d None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:01:24.511440 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e18789-ef3f-4de2-a791-bf6ddcb1cf7d None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:01:24.511440 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e18789-ef3f-4de2-a791-bf6ddcb1cf7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:01:24.511440 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-16e18789-ef3f-4de2-a791-bf6ddcb1cf7d None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:01:24.511440 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-16e18789-ef3f-4de2-a791-bf6ddcb1cf7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:01:24.568264 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 650/1297] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Aug 24 06:01:24 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 06:01:24.831272 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b643a54-cf5a-4e3e-b3d6-8e3f552922d8 None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:01:24.929042 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b643a54-cf5a-4e3e-b3d6-8e3f552922d8 None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:01:24.961877 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fc6ff8-0a8b-4ab7-b8c7-446af7b2902e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:01:24.962484 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fc6ff8-0a8b-4ab7-b8c7-446af7b2902e None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:01:24.962756 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fc6ff8-0a8b-4ab7-b8c7-446af7b2902e None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:01:24.963020 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fc6ff8-0a8b-4ab7-b8c7-446af7b2902e None admin] PATH_INFO: `/v3/services/placement` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:01:24.963708 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fc6ff8-0a8b-4ab7-b8c7-446af7b2902e None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:01:24.964967 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fc6ff8-0a8b-4ab7-b8c7-446af7b2902e None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:01:24.973957 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application [None req-31fc6ff8-0a8b-4ab7-b8c7-446af7b2902e None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 06:01:24.973957 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:01:24.973957 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 06:01:24.973957 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 06:01:24.973957 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 06:01:24.973957 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 06:01:24.973957 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 06:01:24.973957 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 06:01:24.973957 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 06:01:24.973957 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Aug 24 06:01:24.973957 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 06:01:24.973957 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Aug 24 06:01:24.973957 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:01:24.973957 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:01:24.973957 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:01:24.973957 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:01:24.973957 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:01:24.975313 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:01:24.975313 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:01:24.975313 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:01:24.975313 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:01:24.975313 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:01:24.975313 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:01:24.975313 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 06:01:24.975313 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 06:01:24.975313 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 06:01:24.975313 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 06:01:24.975313 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:01:24.975313 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:01:24.975313 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:01:24.975313 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:01:24.975313 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:01:24.975313 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:01:24.976515 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:01:24.976515 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application with Lock( Aug 24 06:01:24.976515 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:01:24.976515 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:01:24.976515 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:01:24.976515 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:01:24.976515 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:01:24.976515 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:01:24.976515 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:01:24.976515 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:01:24.976515 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 06:01:24.976515 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 06:01:24.976515 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 06:01:24.976515 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Aug 24 06:01:24.976515 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 648/1298] 10.222.0.62 () {56 vars in 1117 bytes} [Wed Aug 24 06:01:24 2022] GET /identity/v3/services/placement => generated 90 bytes in 145 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:01:24.981367 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e18789-ef3f-4de2-a791-bf6ddcb1cf7d None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:01:24.999160 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:01:25.083164 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e18789-ef3f-4de2-a791-bf6ddcb1cf7d None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:01:25.115310 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9cec47-001b-4a13-b911-acb89c6b3634 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:01:25.116338 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9cec47-001b-4a13-b911-acb89c6b3634 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:01:25.116338 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9cec47-001b-4a13-b911-acb89c6b3634 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:01:25.116538 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9cec47-001b-4a13-b911-acb89c6b3634 None admin] PATH_INFO: `/v3/services` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:01:25.117050 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9cec47-001b-4a13-b911-acb89c6b3634 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:01:25.117348 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9cec47-001b-4a13-b911-acb89c6b3634 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:01:25.118689 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9cec47-001b-4a13-b911-acb89c6b3634 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:01:25.123641 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 651/1299] 10.222.0.62 () {56 vars in 1116 bytes} [Wed Aug 24 06:01:24 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:01:25.129671 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fc6ff8-0a8b-4ab7-b8c7-446af7b2902e None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:01:25.162646 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:01:25.229785 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fc6ff8-0a8b-4ab7-b8c7-446af7b2902e None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:01:25.262529 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2dbc84c-922d-4302-b981-e472a5998f84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:01:25.263118 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dbc84c-922d-4302-b981-e472a5998f84 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:01:25.263395 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dbc84c-922d-4302-b981-e472a5998f84 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:01:25.263667 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dbc84c-922d-4302-b981-e472a5998f84 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:01:25.264992 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2dbc84c-922d-4302-b981-e472a5998f84 None admin] RBAC: Adding query filter params (interface=public, service_id=eca5add261fc4022b8913d14cbf20e4c, region_id=RegionOne) {{(pid=81851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:01:25.265300 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2dbc84c-922d-4302-b981-e472a5998f84 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:01:25.266662 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2dbc84c-922d-4302-b981-e472a5998f84 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:01:25.271648 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 649/1300] 10.222.0.62 () {56 vars in 1251 bytes} [Wed Aug 24 06:01:25 2022] GET /identity/v3/endpoints?service_id=eca5add261fc4022b8913d14cbf20e4c&interface=public®ion_id=RegionOne => generated 194 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:01:25.277369 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9cec47-001b-4a13-b911-acb89c6b3634 None admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:01:25.355196 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:01:25.371708 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9cec47-001b-4a13-b911-acb89c6b3634 None admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:01:25.404844 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000d852f-b316-4e4f-86d4-e5fd2be3186e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:01:25.405433 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000d852f-b316-4e4f-86d4-e5fd2be3186e None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:01:25.405704 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000d852f-b316-4e4f-86d4-e5fd2be3186e None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:01:25.405968 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000d852f-b316-4e4f-86d4-e5fd2be3186e None admin] PATH_INFO: `/v3/services` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:01:25.406549 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000d852f-b316-4e4f-86d4-e5fd2be3186e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:01:25.407891 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000d852f-b316-4e4f-86d4-e5fd2be3186e None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:01:25.414392 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 652/1301] 10.222.0.62 () {56 vars in 1087 bytes} [Wed Aug 24 06:01:25 2022] GET /identity/v3/services => generated 2191 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:01:27.287597 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e217e1-cf82-4f9c-936b-3c0ec520af07 None None] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:01:27.288157 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e217e1-cf82-4f9c-936b-3c0ec520af07 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:01:27.288157 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e217e1-cf82-4f9c-936b-3c0ec520af07 None None] PATH_INFO: `/` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:01:27.289540 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 650/1302] 10.222.0.62 () {54 vars in 927 bytes} [Wed Aug 24 06:01:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 06:01:27.295172 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc645aea-3c05-40d3-9c8f-c89c0e8a41bc None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:01:27.296100 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc645aea-3c05-40d3-9c8f-c89c0e8a41bc None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:01:27.296240 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc645aea-3c05-40d3-9c8f-c89c0e8a41bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:01:27.314548 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-fc645aea-3c05-40d3-9c8f-c89c0e8a41bc None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:01:27.315715 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-fc645aea-3c05-40d3-9c8f-c89c0e8a41bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:01:27.379997 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 653/1303] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Aug 24 06:01:27 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 06:01:27.651335 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e217e1-cf82-4f9c-936b-3c0ec520af07 None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:01:27.744364 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:01:27.751426 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e217e1-cf82-4f9c-936b-3c0ec520af07 None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:01:27.783653 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4aeb0af-676d-421b-a57f-4ea1cded2ca1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:01:27.784261 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4aeb0af-676d-421b-a57f-4ea1cded2ca1 None admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:01:27.784678 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4aeb0af-676d-421b-a57f-4ea1cded2ca1 None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:01:27.784951 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4aeb0af-676d-421b-a57f-4ea1cded2ca1 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:01:27.785517 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4aeb0af-676d-421b-a57f-4ea1cded2ca1 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:01:27.786950 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4aeb0af-676d-421b-a57f-4ea1cded2ca1 None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:01:27.796086 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application [None req-c4aeb0af-676d-421b-a57f-4ea1cded2ca1 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 06:01:27.796086 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:01:27.796086 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 06:01:27.796086 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 06:01:27.796086 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 06:01:27.796086 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 06:01:27.796086 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 06:01:27.796086 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 06:01:27.796086 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 06:01:27.796086 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Aug 24 06:01:27.796086 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 06:01:27.796086 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Aug 24 06:01:27.796086 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:01:27.796086 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:01:27.796086 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:01:27.796086 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:01:27.796086 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:01:27.797426 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:01:27.797426 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:01:27.797426 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:01:27.797426 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:01:27.797426 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:01:27.797426 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:01:27.797426 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 06:01:27.797426 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 06:01:27.797426 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 06:01:27.797426 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 06:01:27.797426 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:01:27.797426 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:01:27.797426 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:01:27.797426 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:01:27.797426 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:01:27.797426 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:01:27.798604 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:01:27.798604 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application with Lock( Aug 24 06:01:27.798604 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:01:27.798604 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:01:27.798604 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:01:27.798604 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:01:27.798604 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:01:27.798604 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:01:27.798604 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:01:27.798604 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:01:27.798604 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 06:01:27.798604 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 06:01:27.798604 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 06:01:27.798604 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: ERROR keystone.server.flask.application Aug 24 06:01:27.798604 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 651/1304] 10.222.0.62 () {56 vars in 1117 bytes} [Wed Aug 24 06:01:27 2022] GET /identity/v3/services/placement => generated 90 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:01:27.803313 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc645aea-3c05-40d3-9c8f-c89c0e8a41bc None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:01:27.876601 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:01:27.898106 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc645aea-3c05-40d3-9c8f-c89c0e8a41bc None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:01:27.930753 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988851a4-3990-4450-a9d9-a89e37427c61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:01:27.931347 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988851a4-3990-4450-a9d9-a89e37427c61 None admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:01:27.931621 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988851a4-3990-4450-a9d9-a89e37427c61 None admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:01:27.931888 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988851a4-3990-4450-a9d9-a89e37427c61 None admin] PATH_INFO: `/v3/services` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:01:27.932522 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988851a4-3990-4450-a9d9-a89e37427c61 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:01:27.932826 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988851a4-3990-4450-a9d9-a89e37427c61 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:01:27.934210 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988851a4-3990-4450-a9d9-a89e37427c61 None admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:01:27.939739 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 654/1305] 10.222.0.62 () {56 vars in 1116 bytes} [Wed Aug 24 06:01:27 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:01:27.946896 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4aeb0af-676d-421b-a57f-4ea1cded2ca1 None admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:01:28.042670 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4aeb0af-676d-421b-a57f-4ea1cded2ca1 None admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:01:28.076030 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7144a28d-767f-4998-9837-24580906240d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:01:28.076706 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7144a28d-767f-4998-9837-24580906240d None admin] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:01:28.076986 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7144a28d-767f-4998-9837-24580906240d None admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:01:28.077253 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7144a28d-767f-4998-9837-24580906240d None admin] PATH_INFO: `/v3/endpoints` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:01:28.077951 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7144a28d-767f-4998-9837-24580906240d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:01:28.079486 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7144a28d-767f-4998-9837-24580906240d None admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:01:28.164297 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 652/1306] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Aug 24 06:01:27 2022] POST /identity/v3/endpoints => generated 331 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:01:38.020997 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=30 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:01:38.159634 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:08:17.698466 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e05ae5-6ee3-4da8-906c-bbb6c0d11c6b None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:08:17.699056 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e05ae5-6ee3-4da8-906c-bbb6c0d11c6b None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:08:17.699056 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e05ae5-6ee3-4da8-906c-bbb6c0d11c6b None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:08:17.700465 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 655/1307] 10.222.0.62 () {54 vars in 927 bytes} [Wed Aug 24 06:08:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:08:17.706463 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c243ba-aae0-4b8f-8f56-d2c2a959b57a None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:08:17.711464 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c243ba-aae0-4b8f-8f56-d2c2a959b57a None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:08:17.711464 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c243ba-aae0-4b8f-8f56-d2c2a959b57a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:08:17.800319 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-03c243ba-aae0-4b8f-8f56-d2c2a959b57a None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:08:17.801751 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-03c243ba-aae0-4b8f-8f56-d2c2a959b57a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:08:17.978974 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 653/1308] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Aug 24 06:08:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:08:18.306478 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053433e0-6276-45b2-839f-0062b3acee4a None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:08:18.307052 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053433e0-6276-45b2-839f-0062b3acee4a None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:08:18.307489 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053433e0-6276-45b2-839f-0062b3acee4a None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:08:18.309296 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 656/1309] 10.222.0.62 () {54 vars in 965 bytes} [Wed Aug 24 06:08:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:08:18.316351 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a87222-9be7-4d43-83e1-bbdf9550b93c None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:08:18.316866 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a87222-9be7-4d43-83e1-bbdf9550b93c None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:08:18.317494 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a87222-9be7-4d43-83e1-bbdf9550b93c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:08:18.412085 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-94a87222-9be7-4d43-83e1-bbdf9550b93c None None] MFA Rules not processed for user `0bb6c79db75e4d169331c02133d1ec32`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:08:18.413595 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-94a87222-9be7-4d43-83e1-bbdf9550b93c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:08:18.439489 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:08:18.515735 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 654/1310] 10.222.0.62 () {58 vars in 1062 bytes} [Wed Aug 24 06:08:18 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:08:18.516734 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053433e0-6276-45b2-839f-0062b3acee4a None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:08:18.518582 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-053433e0-6276-45b2-839f-0062b3acee4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:08:18.654965 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053433e0-6276-45b2-839f-0062b3acee4a None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:08:18.749886 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04952dd-da94-446a-8ddd-8f0192d1bef1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bb6c79db75e4d169331c02133d1ec32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:08:18.751300 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04952dd-da94-446a-8ddd-8f0192d1bef1 service neutron] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:08:18.751732 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04952dd-da94-446a-8ddd-8f0192d1bef1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:08:18.752153 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04952dd-da94-446a-8ddd-8f0192d1bef1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:08:18.754152 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-f04952dd-da94-446a-8ddd-8f0192d1bef1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:08:18.825641 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:08:18.896987 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04952dd-da94-446a-8ddd-8f0192d1bef1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:08:18.898711 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04952dd-da94-446a-8ddd-8f0192d1bef1 service neutron] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:08:18.921379 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 657/1311] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:08:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 06:08:21.225336 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bc0f3b-d545-4d06-b047-bb412fb240a2 None None] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:08:21.225944 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bc0f3b-d545-4d06-b047-bb412fb240a2 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:08:21.226069 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bc0f3b-d545-4d06-b047-bb412fb240a2 None None] PATH_INFO: `/` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:08:21.227444 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 655/1312] 10.222.0.62 () {54 vars in 927 bytes} [Wed Aug 24 06:08:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:08:21.233411 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d76081d-24b3-4f62-9857-6718039d3e32 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:08:21.233826 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d76081d-24b3-4f62-9857-6718039d3e32 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:08:21.234046 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d76081d-24b3-4f62-9857-6718039d3e32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:08:21.317435 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-7d76081d-24b3-4f62-9857-6718039d3e32 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:08:21.318653 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-7d76081d-24b3-4f62-9857-6718039d3e32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:08:21.416574 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 658/1313] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Aug 24 06:08:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:08:21.700080 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bc0f3b-d545-4d06-b047-bb412fb240a2 None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:08:21.701647 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-b9bc0f3b-d545-4d06-b047-bb412fb240a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:08:21.805661 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bc0f3b-d545-4d06-b047-bb412fb240a2 None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:08:21.876120 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:08:21.878680 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea63efa3-5fcf-4231-9cbc-449eea55cf19 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bb6c79db75e4d169331c02133d1ec32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:08:21.879276 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea63efa3-5fcf-4231-9cbc-449eea55cf19 service neutron] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:08:21.879553 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea63efa3-5fcf-4231-9cbc-449eea55cf19 service neutron] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:08:21.879816 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea63efa3-5fcf-4231-9cbc-449eea55cf19 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:08:21.881426 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-ea63efa3-5fcf-4231-9cbc-449eea55cf19 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:08:21.990089 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea63efa3-5fcf-4231-9cbc-449eea55cf19 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:08:21.991567 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea63efa3-5fcf-4231-9cbc-449eea55cf19 service neutron] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:08:22.009425 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 656/1314] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:08:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:08:24.479624 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a428a5e0-25cb-41b9-9711-a5785f561ce5 None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:08:24.480205 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a428a5e0-25cb-41b9-9711-a5785f561ce5 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:08:24.480205 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a428a5e0-25cb-41b9-9711-a5785f561ce5 None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:08:24.481541 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 659/1315] 10.222.0.62 () {54 vars in 927 bytes} [Wed Aug 24 06:08:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:08:24.487055 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c170531-8443-416f-83a8-9187ac240660 None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:08:24.487465 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c170531-8443-416f-83a8-9187ac240660 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:08:24.487612 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c170531-8443-416f-83a8-9187ac240660 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:08:24.553217 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-4c170531-8443-416f-83a8-9187ac240660 None None] MFA Rules not processed for user `f16ca5f6955045a5811c25a8e6cc8055`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:08:24.554415 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-4c170531-8443-416f-83a8-9187ac240660 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:08:24.605398 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:08:24.649887 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 657/1316] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Aug 24 06:08:24 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:08:24.933182 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a428a5e0-25cb-41b9-9711-a5785f561ce5 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:08:24.934632 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-a428a5e0-25cb-41b9-9711-a5785f561ce5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:08:24.945044 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:08:25.035950 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a428a5e0-25cb-41b9-9711-a5785f561ce5 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:08:25.107517 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a45238-333c-422a-a9d1-e7228143bf8d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bb6c79db75e4d169331c02133d1ec32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:08:25.108116 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a45238-333c-422a-a9d1-e7228143bf8d service neutron] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:08:25.108394 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a45238-333c-422a-a9d1-e7228143bf8d service neutron] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:08:25.108698 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a45238-333c-422a-a9d1-e7228143bf8d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:08:25.110824 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-05a45238-333c-422a-a9d1-e7228143bf8d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:08:25.171710 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:08:25.219551 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a45238-333c-422a-a9d1-e7228143bf8d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:08:25.220943 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a45238-333c-422a-a9d1-e7228143bf8d service neutron] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:08:25.238611 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 660/1317] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:08:24 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:08:27.949292 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df79635-a72c-4f1a-ba5e-81e050a1255a None None] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:08:27.949889 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df79635-a72c-4f1a-ba5e-81e050a1255a None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:08:27.949889 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df79635-a72c-4f1a-ba5e-81e050a1255a None None] PATH_INFO: `/` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:08:27.951178 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 658/1318] 10.222.0.62 () {54 vars in 927 bytes} [Wed Aug 24 06:08:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:08:27.957064 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c42311-9ced-4b87-9b8e-cbdb4c4418a4 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:08:27.957508 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c42311-9ced-4b87-9b8e-cbdb4c4418a4 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:08:27.957605 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c42311-9ced-4b87-9b8e-cbdb4c4418a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:08:28.021708 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-51c42311-9ced-4b87-9b8e-cbdb4c4418a4 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:08:28.022902 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-51c42311-9ced-4b87-9b8e-cbdb4c4418a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:08:28.117309 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 661/1319] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Aug 24 06:08:27 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:08:28.402763 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df79635-a72c-4f1a-ba5e-81e050a1255a None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:08:28.404329 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-3df79635-a72c-4f1a-ba5e-81e050a1255a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:08:28.505884 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df79635-a72c-4f1a-ba5e-81e050a1255a None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:08:28.580512 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64950809-7da0-4111-a1c5-eeba5ebaa92b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bb6c79db75e4d169331c02133d1ec32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:08:28.581113 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64950809-7da0-4111-a1c5-eeba5ebaa92b service neutron] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:08:28.581389 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64950809-7da0-4111-a1c5-eeba5ebaa92b service neutron] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:08:28.581654 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64950809-7da0-4111-a1c5-eeba5ebaa92b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:08:28.583222 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-64950809-7da0-4111-a1c5-eeba5ebaa92b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:08:28.590527 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:08:28.691202 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64950809-7da0-4111-a1c5-eeba5ebaa92b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:08:28.692668 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64950809-7da0-4111-a1c5-eeba5ebaa92b service neutron] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:08:28.711591 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 659/1320] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:08:28 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:08:31.575184 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3203c3-8cea-4c50-af2c-857105f7f169 None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:08:31.575748 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3203c3-8cea-4c50-af2c-857105f7f169 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:08:31.575748 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3203c3-8cea-4c50-af2c-857105f7f169 None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:08:31.577130 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 662/1321] 10.222.0.62 () {54 vars in 927 bytes} [Wed Aug 24 06:08:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:08:31.582541 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d230162c-306d-4637-aee0-7831549ec4b7 None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:08:31.582944 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d230162c-306d-4637-aee0-7831549ec4b7 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:08:31.583217 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d230162c-306d-4637-aee0-7831549ec4b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:08:31.652595 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-d230162c-306d-4637-aee0-7831549ec4b7 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:08:31.653820 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-d230162c-306d-4637-aee0-7831549ec4b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:08:31.717725 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:08:31.757012 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 660/1322] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Aug 24 06:08:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:08:32.055462 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7183ba92-dec0-45c8-ba8d-b02b920a1817 None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:08:32.055749 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7183ba92-dec0-45c8-ba8d-b02b920a1817 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:08:32.056023 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7183ba92-dec0-45c8-ba8d-b02b920a1817 None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:08:32.057570 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 663/1323] 10.222.0.62 () {54 vars in 965 bytes} [Wed Aug 24 06:08:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:08:32.063874 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730ba4d7-66c7-47f3-92e6-9cf4de96b8fc None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:08:32.064267 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730ba4d7-66c7-47f3-92e6-9cf4de96b8fc None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:08:32.064637 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730ba4d7-66c7-47f3-92e6-9cf4de96b8fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:08:32.143765 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-730ba4d7-66c7-47f3-92e6-9cf4de96b8fc None None] MFA Rules not processed for user `0bb6c79db75e4d169331c02133d1ec32`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:08:32.145004 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-730ba4d7-66c7-47f3-92e6-9cf4de96b8fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:08:32.224550 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 661/1324] 10.222.0.62 () {58 vars in 1062 bytes} [Wed Aug 24 06:08:32 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:08:32.233493 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7183ba92-dec0-45c8-ba8d-b02b920a1817 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:08:32.235702 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-7183ba92-dec0-45c8-ba8d-b02b920a1817 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:08:32.255008 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:08:32.338968 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7183ba92-dec0-45c8-ba8d-b02b920a1817 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:08:32.411843 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7e69b1-10f1-4289-a457-aa854169f9de service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bb6c79db75e4d169331c02133d1ec32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:08:32.412443 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7e69b1-10f1-4289-a457-aa854169f9de service neutron] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:08:32.412749 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7e69b1-10f1-4289-a457-aa854169f9de service neutron] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:08:32.413020 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7e69b1-10f1-4289-a457-aa854169f9de service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:08:32.414599 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-8b7e69b1-10f1-4289-a457-aa854169f9de service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:08:32.489047 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:08:32.526192 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7e69b1-10f1-4289-a457-aa854169f9de service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:08:32.528335 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7e69b1-10f1-4289-a457-aa854169f9de service neutron] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:08:32.545835 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 664/1325] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:08:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:08:34.785552 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07893df-b324-4b2c-89ab-ba431fac5b5f None None] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:08:34.786110 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07893df-b324-4b2c-89ab-ba431fac5b5f None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:08:34.786110 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07893df-b324-4b2c-89ab-ba431fac5b5f None None] PATH_INFO: `/` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:08:34.787470 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 662/1326] 10.222.0.62 () {54 vars in 927 bytes} [Wed Aug 24 06:08:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:08:34.793884 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e12f0e-a459-4502-b1d7-6a43c0adc39e None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:08:34.794479 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e12f0e-a459-4502-b1d7-6a43c0adc39e None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:08:34.794479 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e12f0e-a459-4502-b1d7-6a43c0adc39e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:08:34.864606 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-f5e12f0e-a459-4502-b1d7-6a43c0adc39e None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:08:34.865823 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-f5e12f0e-a459-4502-b1d7-6a43c0adc39e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:08:34.960711 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 665/1327] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Aug 24 06:08:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:08:35.248256 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07893df-b324-4b2c-89ab-ba431fac5b5f None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:08:35.252962 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-f07893df-b324-4b2c-89ab-ba431fac5b5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:08:35.314164 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:08:35.357612 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07893df-b324-4b2c-89ab-ba431fac5b5f None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:08:35.432732 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d1748b-2d8f-43c8-bcdf-8d897d44e18d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bb6c79db75e4d169331c02133d1ec32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:08:35.433348 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d1748b-2d8f-43c8-bcdf-8d897d44e18d service neutron] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:08:35.433628 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d1748b-2d8f-43c8-bcdf-8d897d44e18d service neutron] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:08:35.433904 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d1748b-2d8f-43c8-bcdf-8d897d44e18d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:08:35.435563 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-87d1748b-2d8f-43c8-bcdf-8d897d44e18d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:08:35.545199 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:08:35.547926 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d1748b-2d8f-43c8-bcdf-8d897d44e18d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:08:35.549848 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d1748b-2d8f-43c8-bcdf-8d897d44e18d service neutron] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:08:35.569026 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 663/1328] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:08:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:08:37.580722 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2aff7b2-68ad-45f1-ab55-9914720a24cc None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:08:37.581264 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2aff7b2-68ad-45f1-ab55-9914720a24cc None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:08:37.581264 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2aff7b2-68ad-45f1-ab55-9914720a24cc None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:08:37.582618 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 666/1329] 10.222.0.62 () {54 vars in 927 bytes} [Wed Aug 24 06:08:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:08:37.587801 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630b3b87-ca36-423e-a820-6de8b01656c7 None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:08:37.588272 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630b3b87-ca36-423e-a820-6de8b01656c7 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:08:37.588370 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630b3b87-ca36-423e-a820-6de8b01656c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:08:37.674814 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-630b3b87-ca36-423e-a820-6de8b01656c7 None None] MFA Rules not processed for user `f16ca5f6955045a5811c25a8e6cc8055`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:08:37.677435 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-630b3b87-ca36-423e-a820-6de8b01656c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:08:37.779104 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 664/1330] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Aug 24 06:08:37 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:08:38.069011 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2aff7b2-68ad-45f1-ab55-9914720a24cc None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:08:38.070545 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-d2aff7b2-68ad-45f1-ab55-9914720a24cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:08:38.102439 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:08:38.177102 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2aff7b2-68ad-45f1-ab55-9914720a24cc None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:08:38.251823 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73775161-0720-4614-8790-a1d2bd38acc1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bb6c79db75e4d169331c02133d1ec32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:08:38.252435 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73775161-0720-4614-8790-a1d2bd38acc1 service neutron] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:08:38.252764 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73775161-0720-4614-8790-a1d2bd38acc1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:08:38.253019 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73775161-0720-4614-8790-a1d2bd38acc1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:08:38.254657 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-73775161-0720-4614-8790-a1d2bd38acc1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:08:38.336756 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:08:38.369453 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73775161-0720-4614-8790-a1d2bd38acc1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:08:38.370885 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73775161-0720-4614-8790-a1d2bd38acc1 service neutron] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:08:38.389805 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 667/1331] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:08:38 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:08:40.948815 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5792bff-cde2-4f51-8ec7-d189380b026a None None] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:08:40.949381 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5792bff-cde2-4f51-8ec7-d189380b026a None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:08:40.949381 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5792bff-cde2-4f51-8ec7-d189380b026a None None] PATH_INFO: `/` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:08:40.950670 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 665/1332] 10.222.0.62 () {54 vars in 927 bytes} [Wed Aug 24 06:08:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:08:40.955876 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d879d4-ca6c-4b84-a9e1-6c4d67fee134 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:08:40.956316 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d879d4-ca6c-4b84-a9e1-6c4d67fee134 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:08:40.956417 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d879d4-ca6c-4b84-a9e1-6c4d67fee134 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:08:41.020949 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-b0d879d4-ca6c-4b84-a9e1-6c4d67fee134 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:08:41.022040 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-b0d879d4-ca6c-4b84-a9e1-6c4d67fee134 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:08:41.125959 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 668/1333] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Aug 24 06:08:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 06:08:41.408471 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5792bff-cde2-4f51-8ec7-d189380b026a None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:08:41.410085 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-c5792bff-cde2-4f51-8ec7-d189380b026a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:08:41.479276 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:08:41.513051 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5792bff-cde2-4f51-8ec7-d189380b026a None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:08:41.585907 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36234abc-b61a-4390-82f5-af70d3dfe901 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bb6c79db75e4d169331c02133d1ec32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:08:41.586511 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36234abc-b61a-4390-82f5-af70d3dfe901 service neutron] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:08:41.586791 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36234abc-b61a-4390-82f5-af70d3dfe901 service neutron] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:08:41.587062 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36234abc-b61a-4390-82f5-af70d3dfe901 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:08:41.588672 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-36234abc-b61a-4390-82f5-af70d3dfe901 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:08:41.695798 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36234abc-b61a-4390-82f5-af70d3dfe901 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:08:41.697269 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36234abc-b61a-4390-82f5-af70d3dfe901 service neutron] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:08:41.708164 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:08:41.717523 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 666/1334] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:08:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:08:43.674053 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6bccd4-9295-492a-b53f-dee9b49b11f9 None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:08:43.674618 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6bccd4-9295-492a-b53f-dee9b49b11f9 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:08:43.674618 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6bccd4-9295-492a-b53f-dee9b49b11f9 None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:08:43.676241 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 669/1335] 10.222.0.62 () {54 vars in 927 bytes} [Wed Aug 24 06:08:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:08:43.681575 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6816b8e3-4236-4f5d-97a0-5ab4d92a96b3 None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:08:43.681962 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6816b8e3-4236-4f5d-97a0-5ab4d92a96b3 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:08:43.682122 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6816b8e3-4236-4f5d-97a0-5ab4d92a96b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:08:43.746710 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-6816b8e3-4236-4f5d-97a0-5ab4d92a96b3 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:08:43.747836 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-6816b8e3-4236-4f5d-97a0-5ab4d92a96b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:08:43.842687 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 667/1336] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Aug 24 06:08:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:08:44.130968 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6bccd4-9295-492a-b53f-dee9b49b11f9 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:08:44.132513 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-0b6bccd4-9295-492a-b53f-dee9b49b11f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:08:44.171965 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:08:44.235974 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6bccd4-9295-492a-b53f-dee9b49b11f9 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:08:44.309573 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4beb4eae-4721-4e0e-9e2b-39b2bd444f6e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bb6c79db75e4d169331c02133d1ec32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:08:44.310174 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4beb4eae-4721-4e0e-9e2b-39b2bd444f6e service neutron] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:08:44.310448 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4beb4eae-4721-4e0e-9e2b-39b2bd444f6e service neutron] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:08:44.310716 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4beb4eae-4721-4e0e-9e2b-39b2bd444f6e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:08:44.312631 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-4beb4eae-4721-4e0e-9e2b-39b2bd444f6e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:08:44.394514 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:08:44.420820 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4beb4eae-4721-4e0e-9e2b-39b2bd444f6e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:08:44.422314 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4beb4eae-4721-4e0e-9e2b-39b2bd444f6e service neutron] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:08:44.440858 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 670/1337] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:08:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:08:46.921400 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce48720-7f9a-46fc-864f-ff2e8d94b8cc None None] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:08:46.921937 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce48720-7f9a-46fc-864f-ff2e8d94b8cc None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:08:46.921937 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce48720-7f9a-46fc-864f-ff2e8d94b8cc None None] PATH_INFO: `/` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:08:46.923269 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 668/1338] 10.222.0.62 () {54 vars in 927 bytes} [Wed Aug 24 06:08:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:08:46.928698 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee2845a-69a8-4106-a706-5b22cf8f7cca None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:08:46.929113 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee2845a-69a8-4106-a706-5b22cf8f7cca None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:08:46.929249 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee2845a-69a8-4106-a706-5b22cf8f7cca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:08:46.998554 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-fee2845a-69a8-4106-a706-5b22cf8f7cca None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:08:46.999669 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-fee2845a-69a8-4106-a706-5b22cf8f7cca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:08:47.100038 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 671/1339] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Aug 24 06:08:46 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:08:47.385357 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce48720-7f9a-46fc-864f-ff2e8d94b8cc None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:08:47.386906 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-9ce48720-7f9a-46fc-864f-ff2e8d94b8cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:08:47.462292 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:08:47.489643 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce48720-7f9a-46fc-864f-ff2e8d94b8cc None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:08:47.562324 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada77bfa-f6fd-4112-b8b1-b91bba61d032 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bb6c79db75e4d169331c02133d1ec32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:08:47.562923 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada77bfa-f6fd-4112-b8b1-b91bba61d032 service neutron] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:08:47.563199 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada77bfa-f6fd-4112-b8b1-b91bba61d032 service neutron] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:08:47.563471 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada77bfa-f6fd-4112-b8b1-b91bba61d032 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:08:47.565113 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-ada77bfa-f6fd-4112-b8b1-b91bba61d032 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:08:47.671010 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada77bfa-f6fd-4112-b8b1-b91bba61d032 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:08:47.672427 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada77bfa-f6fd-4112-b8b1-b91bba61d032 service neutron] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:08:47.692417 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 669/1340] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:08:47 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:08:51.854126 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cacd62-f0ee-48f0-af03-500b79e89daa None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:08:51.854651 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cacd62-f0ee-48f0-af03-500b79e89daa None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:08:51.854731 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cacd62-f0ee-48f0-af03-500b79e89daa None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:08:51.856026 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 672/1341] 10.222.0.62 () {54 vars in 927 bytes} [Wed Aug 24 06:08:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 06:08:51.861658 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7feaa26-0435-4ad8-b715-dc19bff7064a None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:08:51.862160 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7feaa26-0435-4ad8-b715-dc19bff7064a None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:08:51.862160 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7feaa26-0435-4ad8-b715-dc19bff7064a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:08:51.869056 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:08:51.932854 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-a7feaa26-0435-4ad8-b715-dc19bff7064a None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:08:51.934062 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-a7feaa26-0435-4ad8-b715-dc19bff7064a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:08:52.036003 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 670/1342] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Aug 24 06:08:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:08:52.323487 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cacd62-f0ee-48f0-af03-500b79e89daa None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:08:52.325063 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-21cacd62-f0ee-48f0-af03-500b79e89daa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:08:52.373540 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:08:52.429937 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cacd62-f0ee-48f0-af03-500b79e89daa None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:08:52.505453 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab520ce4-17d6-4163-bde5-1b1946833a45 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bb6c79db75e4d169331c02133d1ec32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:08:52.506059 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab520ce4-17d6-4163-bde5-1b1946833a45 service neutron] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:08:52.506335 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab520ce4-17d6-4163-bde5-1b1946833a45 service neutron] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:08:52.506604 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab520ce4-17d6-4163-bde5-1b1946833a45 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:08:52.508187 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-ab520ce4-17d6-4163-bde5-1b1946833a45 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:08:52.603352 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:08:52.619894 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab520ce4-17d6-4163-bde5-1b1946833a45 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:08:52.621375 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab520ce4-17d6-4163-bde5-1b1946833a45 service neutron] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:08:52.639719 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 673/1343] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:08:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:08:55.739521 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf42ad6a-25f9-419c-b12f-ff052de1613d None None] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:08:55.740058 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf42ad6a-25f9-419c-b12f-ff052de1613d None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:08:55.740168 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf42ad6a-25f9-419c-b12f-ff052de1613d None None] PATH_INFO: `/` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:08:55.741509 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 671/1344] 10.222.0.62 () {54 vars in 927 bytes} [Wed Aug 24 06:08:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:08:55.746994 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c7bff5-2783-4218-97f8-5fcc6ec2530c None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:08:55.747449 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c7bff5-2783-4218-97f8-5fcc6ec2530c None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:08:55.747540 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c7bff5-2783-4218-97f8-5fcc6ec2530c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:08:55.815848 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-a9c7bff5-2783-4218-97f8-5fcc6ec2530c None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:08:55.817153 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-a9c7bff5-2783-4218-97f8-5fcc6ec2530c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:08:55.914564 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 674/1345] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Aug 24 06:08:55 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:08:56.203145 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf42ad6a-25f9-419c-b12f-ff052de1613d None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:08:56.205538 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-bf42ad6a-25f9-419c-b12f-ff052de1613d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:08:56.293501 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:08:56.311223 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf42ad6a-25f9-419c-b12f-ff052de1613d None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:08:56.391283 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cf385f-523e-47e1-88b1-db38ca6917e0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bb6c79db75e4d169331c02133d1ec32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:08:56.391933 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cf385f-523e-47e1-88b1-db38ca6917e0 service neutron] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:08:56.392189 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cf385f-523e-47e1-88b1-db38ca6917e0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:08:56.392489 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cf385f-523e-47e1-88b1-db38ca6917e0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:08:56.394537 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-29cf385f-523e-47e1-88b1-db38ca6917e0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:08:56.513288 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29cf385f-523e-47e1-88b1-db38ca6917e0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:08:56.515269 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29cf385f-523e-47e1-88b1-db38ca6917e0 service neutron] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:08:56.535197 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 672/1346] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:08:56 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:09:01.043333 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e735e3b9-ba34-471f-97af-ee2371f0351d None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:09:01.043871 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e735e3b9-ba34-471f-97af-ee2371f0351d None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:09:01.043961 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e735e3b9-ba34-471f-97af-ee2371f0351d None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:09:01.045847 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 675/1347] 10.222.0.62 () {54 vars in 927 bytes} [Wed Aug 24 06:09:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:09:01.051785 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9307253-f64a-4b7d-9649-95d746363b81 None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:09:01.052257 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9307253-f64a-4b7d-9649-95d746363b81 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:09:01.052346 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9307253-f64a-4b7d-9649-95d746363b81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:09:01.069510 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:09:01.126578 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-d9307253-f64a-4b7d-9649-95d746363b81 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:09:01.127827 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-d9307253-f64a-4b7d-9649-95d746363b81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:09:01.231957 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 673/1348] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Aug 24 06:09:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:09:01.517397 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e735e3b9-ba34-471f-97af-ee2371f0351d None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:09:01.519180 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-e735e3b9-ba34-471f-97af-ee2371f0351d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:09:01.588726 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:09:01.637228 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e735e3b9-ba34-471f-97af-ee2371f0351d None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:09:01.717361 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fd59e1-cfba-429f-89eb-8b533db92d71 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bb6c79db75e4d169331c02133d1ec32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:09:01.717982 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fd59e1-cfba-429f-89eb-8b533db92d71 service neutron] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:09:01.718250 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fd59e1-cfba-429f-89eb-8b533db92d71 service neutron] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:09:01.718521 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fd59e1-cfba-429f-89eb-8b533db92d71 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:09:01.720130 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-b8fd59e1-cfba-429f-89eb-8b533db92d71 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:09:01.829201 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:09:01.832829 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fd59e1-cfba-429f-89eb-8b533db92d71 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:09:01.834346 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fd59e1-cfba-429f-89eb-8b533db92d71 service neutron] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:09:01.853319 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 676/1349] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:09:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:09:06.730089 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5912338-cfed-471b-a68c-14ee6f6cda7c None None] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:09:06.730623 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5912338-cfed-471b-a68c-14ee6f6cda7c None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:09:06.730623 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5912338-cfed-471b-a68c-14ee6f6cda7c None None] PATH_INFO: `/` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:09:06.733259 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 674/1350] 10.222.0.62 () {54 vars in 921 bytes} [Wed Aug 24 06:09:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:09:06.738227 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f19016-14a2-4be7-989b-01907f4b885e None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:09:06.738705 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f19016-14a2-4be7-989b-01907f4b885e None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:09:06.738855 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f19016-14a2-4be7-989b-01907f4b885e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:09:06.868695 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-d7f19016-14a2-4be7-989b-01907f4b885e None None] MFA Rules not processed for user `abe9d8c9291845dfa4ef55034e5c9e32`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:09:06.870297 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-d7f19016-14a2-4be7-989b-01907f4b885e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:09:07.022742 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 677/1351] 10.222.0.62 () {58 vars in 1018 bytes} [Wed Aug 24 06:09:06 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 285 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:09:11.190014 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ec8f00-8cdf-49cd-a543-bb120129ab4e None None] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:09:11.190632 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ec8f00-8cdf-49cd-a543-bb120129ab4e None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:09:11.190949 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ec8f00-8cdf-49cd-a543-bb120129ab4e None None] PATH_INFO: `/` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:09:11.192394 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 675/1352] 10.222.0.62 () {54 vars in 921 bytes} [Wed Aug 24 06:09:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:09:11.197794 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77df64e-e063-4484-9865-da14c211e431 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:09:11.198243 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77df64e-e063-4484-9865-da14c211e431 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:09:11.198700 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77df64e-e063-4484-9865-da14c211e431 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:09:11.230031 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=59 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:09:11.261224 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:09:11.289565 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-d77df64e-e063-4484-9865-da14c211e431 None None] MFA Rules not processed for user `abe9d8c9291845dfa4ef55034e5c9e32`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:09:11.290776 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-d77df64e-e063-4484-9865-da14c211e431 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:09:11.414454 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 678/1353] 10.222.0.62 () {58 vars in 1018 bytes} [Wed Aug 24 06:09:11 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:09:12.721023 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc76cbf-b5a4-464a-a8ed-265e5e10ab6e None None] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:09:12.721516 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc76cbf-b5a4-464a-a8ed-265e5e10ab6e None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:09:12.721633 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc76cbf-b5a4-464a-a8ed-265e5e10ab6e None None] PATH_INFO: `/` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:09:12.722954 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 676/1354] 10.222.0.62 () {54 vars in 921 bytes} [Wed Aug 24 06:09:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:09:12.728422 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671cf9bb-df6f-44fb-8811-db85920cf2e3 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:09:12.728913 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671cf9bb-df6f-44fb-8811-db85920cf2e3 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:09:12.729021 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671cf9bb-df6f-44fb-8811-db85920cf2e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:09:12.804960 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-671cf9bb-df6f-44fb-8811-db85920cf2e3 None None] MFA Rules not processed for user `abe9d8c9291845dfa4ef55034e5c9e32`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:09:12.806187 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-671cf9bb-df6f-44fb-8811-db85920cf2e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:09:12.848990 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:09:12.914536 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 679/1355] 10.222.0.62 () {58 vars in 1018 bytes} [Wed Aug 24 06:09:12 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:09:16.387463 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620d8f7c-0d79-4cb7-8201-ff3402540edb None None] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:09:16.389219 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620d8f7c-0d79-4cb7-8201-ff3402540edb None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:09:16.389351 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620d8f7c-0d79-4cb7-8201-ff3402540edb None None] PATH_INFO: `/` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:09:16.390666 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 677/1356] 10.222.0.62 () {54 vars in 927 bytes} [Wed Aug 24 06:09:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:09:16.397070 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6d7be4-2fe7-46db-9056-7f1233da05e6 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:09:16.397943 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6d7be4-2fe7-46db-9056-7f1233da05e6 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:09:16.398275 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6d7be4-2fe7-46db-9056-7f1233da05e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:09:16.480827 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-1d6d7be4-2fe7-46db-9056-7f1233da05e6 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:09:16.482424 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-1d6d7be4-2fe7-46db-9056-7f1233da05e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:09:16.606702 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 680/1357] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Aug 24 06:09:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:09:16.917768 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69438d25-7b14-4202-aa01-5a2cfbee3c4c None None] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:09:16.918532 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69438d25-7b14-4202-aa01-5a2cfbee3c4c None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:09:16.920262 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69438d25-7b14-4202-aa01-5a2cfbee3c4c None None] PATH_INFO: `/` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:09:16.923819 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 678/1358] 10.222.0.62 () {54 vars in 962 bytes} [Wed Aug 24 06:09:16 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:09:16.932750 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e624f9-1e72-44e3-9bea-2bcc77945fa6 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:09:16.937614 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e624f9-1e72-44e3-9bea-2bcc77945fa6 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:09:16.937614 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e624f9-1e72-44e3-9bea-2bcc77945fa6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:09:16.950334 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:09:17.019618 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-b9e624f9-1e72-44e3-9bea-2bcc77945fa6 None None] MFA Rules not processed for user `fb6d3743340241f98cacb9e0d063298c`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:09:17.021197 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-b9e624f9-1e72-44e3-9bea-2bcc77945fa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:09:17.150139 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 681/1359] 10.222.0.62 () {58 vars in 1059 bytes} [Wed Aug 24 06:09:16 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 06:09:17.158464 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69438d25-7b14-4202-aa01-5a2cfbee3c4c None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:09:17.159988 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-69438d25-7b14-4202-aa01-5a2cfbee3c4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:09:17.317771 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69438d25-7b14-4202-aa01-5a2cfbee3c4c None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:09:17.394230 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:09:17.437206 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3a9c1e-223c-43a7-83c6-4033b00aa0fb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb6d3743340241f98cacb9e0d063298c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:09:17.438079 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3a9c1e-223c-43a7-83c6-4033b00aa0fb service nova] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:09:17.438518 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3a9c1e-223c-43a7-83c6-4033b00aa0fb service nova] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:09:17.438952 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3a9c1e-223c-43a7-83c6-4033b00aa0fb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:09:17.440961 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-6c3a9c1e-223c-43a7-83c6-4033b00aa0fb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:09:17.540075 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3a9c1e-223c-43a7-83c6-4033b00aa0fb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:09:17.541812 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3a9c1e-223c-43a7-83c6-4033b00aa0fb service nova] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:09:17.562117 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 679/1360] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:09:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:09:19.704936 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d81733-e11f-4931-a4bc-d24aea3322c2 None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:09:19.705452 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d81733-e11f-4931-a4bc-d24aea3322c2 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:09:19.705452 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d81733-e11f-4931-a4bc-d24aea3322c2 None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:09:19.707609 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 682/1361] 10.222.0.62 () {54 vars in 927 bytes} [Wed Aug 24 06:09:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:09:19.712954 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe86c256-bf7b-4016-8186-925874945f2b None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:09:19.713379 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe86c256-bf7b-4016-8186-925874945f2b None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:09:19.713509 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe86c256-bf7b-4016-8186-925874945f2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:09:19.783741 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-fe86c256-bf7b-4016-8186-925874945f2b None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:09:19.784968 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-fe86c256-bf7b-4016-8186-925874945f2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:09:19.825510 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:09:19.920865 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 680/1362] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Aug 24 06:09:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:09:20.204941 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d81733-e11f-4931-a4bc-d24aea3322c2 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:09:20.213360 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-e4d81733-e11f-4931-a4bc-d24aea3322c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:09:20.290799 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:09:20.383329 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d81733-e11f-4931-a4bc-d24aea3322c2 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:09:20.503865 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406eb4e3-713e-40aa-908a-cd1b41a3699c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb6d3743340241f98cacb9e0d063298c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:09:20.504408 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406eb4e3-713e-40aa-908a-cd1b41a3699c service nova] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:09:20.504710 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406eb4e3-713e-40aa-908a-cd1b41a3699c service nova] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:09:20.504990 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406eb4e3-713e-40aa-908a-cd1b41a3699c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:09:20.506629 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-406eb4e3-713e-40aa-908a-cd1b41a3699c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:09:20.531551 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:09:20.613406 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406eb4e3-713e-40aa-908a-cd1b41a3699c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:09:20.615337 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406eb4e3-713e-40aa-908a-cd1b41a3699c service nova] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:09:20.636871 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 683/1363] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:09:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 437 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:09:23.058925 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b48e642-116a-480c-917d-bde1fd0e0338 None None] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:09:23.059461 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b48e642-116a-480c-917d-bde1fd0e0338 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:09:23.059461 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b48e642-116a-480c-917d-bde1fd0e0338 None None] PATH_INFO: `/` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:09:23.061682 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 681/1364] 10.222.0.62 () {54 vars in 927 bytes} [Wed Aug 24 06:09:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:09:23.067200 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc8b012-4cad-46cb-bfba-5f3341c9a5f4 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:09:23.067686 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc8b012-4cad-46cb-bfba-5f3341c9a5f4 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:09:23.067780 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc8b012-4cad-46cb-bfba-5f3341c9a5f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:09:23.140742 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-4bc8b012-4cad-46cb-bfba-5f3341c9a5f4 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:09:23.142051 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-4bc8b012-4cad-46cb-bfba-5f3341c9a5f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:09:23.244199 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:09:23.247171 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 684/1365] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Aug 24 06:09:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:09:23.541253 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3645ab-5c5d-428d-9f91-ea06185ab6e5 None None] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:09:23.541538 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3645ab-5c5d-428d-9f91-ea06185ab6e5 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:09:23.541890 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3645ab-5c5d-428d-9f91-ea06185ab6e5 None None] PATH_INFO: `/` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:09:23.543378 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 682/1366] 10.222.0.62 () {54 vars in 962 bytes} [Wed Aug 24 06:09:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:09:23.549820 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cfe3ce-2153-4f6a-b4e1-dedcbb43ac88 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:09:23.550422 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cfe3ce-2153-4f6a-b4e1-dedcbb43ac88 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:09:23.550790 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cfe3ce-2153-4f6a-b4e1-dedcbb43ac88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:09:23.631597 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-a2cfe3ce-2153-4f6a-b4e1-dedcbb43ac88 None None] MFA Rules not processed for user `fb6d3743340241f98cacb9e0d063298c`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:09:23.632812 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-a2cfe3ce-2153-4f6a-b4e1-dedcbb43ac88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:09:23.769904 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 685/1367] 10.222.0.62 () {58 vars in 1059 bytes} [Wed Aug 24 06:09:23 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:09:23.781928 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3645ab-5c5d-428d-9f91-ea06185ab6e5 None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:09:23.783657 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-ec3645ab-5c5d-428d-9f91-ea06185ab6e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:09:23.948073 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3645ab-5c5d-428d-9f91-ea06185ab6e5 None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:09:23.969051 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:09:24.071946 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2333f2-d02f-4a35-9af0-4b5940fe7334 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb6d3743340241f98cacb9e0d063298c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:09:24.073404 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2333f2-d02f-4a35-9af0-4b5940fe7334 service nova] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:09:24.073909 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2333f2-d02f-4a35-9af0-4b5940fe7334 service nova] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:09:24.074391 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2333f2-d02f-4a35-9af0-4b5940fe7334 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:09:24.076720 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-4a2333f2-d02f-4a35-9af0-4b5940fe7334 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:09:24.173819 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2333f2-d02f-4a35-9af0-4b5940fe7334 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:09:24.176127 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2333f2-d02f-4a35-9af0-4b5940fe7334 service nova] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:09:24.195307 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 683/1368] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:09:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:09:24.655329 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0497510-0d16-4f45-8346-ffdcc15d3e3b None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:09:24.655870 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0497510-0d16-4f45-8346-ffdcc15d3e3b None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:09:24.655870 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0497510-0d16-4f45-8346-ffdcc15d3e3b None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:09:24.657330 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 686/1369] 10.222.0.62 () {54 vars in 927 bytes} [Wed Aug 24 06:09:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:09:24.662481 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c373b285-a49a-48f2-b8c8-701de426379e None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:09:24.662773 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c373b285-a49a-48f2-b8c8-701de426379e None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:09:24.663052 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c373b285-a49a-48f2-b8c8-701de426379e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:09:24.697889 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:09:24.736426 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-c373b285-a49a-48f2-b8c8-701de426379e None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:09:24.738046 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-c373b285-a49a-48f2-b8c8-701de426379e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:09:24.844676 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 684/1370] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Aug 24 06:09:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:09:25.432885 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ee10b7-c2d0-4a4d-9c09-7386586931df None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:09:25.432885 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ee10b7-c2d0-4a4d-9c09-7386586931df None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:09:25.432885 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ee10b7-c2d0-4a4d-9c09-7386586931df None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:09:25.434599 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 687/1371] 10.222.0.62 () {54 vars in 964 bytes} [Wed Aug 24 06:09:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:09:25.438631 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29abc2e1-019a-4514-b7e2-216ff954defa None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:09:25.439168 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29abc2e1-019a-4514-b7e2-216ff954defa None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:09:25.439341 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29abc2e1-019a-4514-b7e2-216ff954defa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:09:25.534112 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-29abc2e1-019a-4514-b7e2-216ff954defa None None] MFA Rules not processed for user `8c4def55ac0e48b29836bc5c64bc7402`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:09:25.535643 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-29abc2e1-019a-4514-b7e2-216ff954defa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:09:25.592398 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:09:25.623730 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 685/1372] 10.222.0.62 () {58 vars in 1061 bytes} [Wed Aug 24 06:09:25 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:09:25.631680 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ee10b7-c2d0-4a4d-9c09-7386586931df None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:09:25.633326 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-a3ee10b7-c2d0-4a4d-9c09-7386586931df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:09:25.689444 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:09:25.754819 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ee10b7-c2d0-4a4d-9c09-7386586931df None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:09:25.835572 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c864e1a-a08c-4bdc-9bea-26831713ab5e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c4def55ac0e48b29836bc5c64bc7402', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:09:25.836214 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c864e1a-a08c-4bdc-9bea-26831713ab5e service cinder] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:09:25.836480 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c864e1a-a08c-4bdc-9bea-26831713ab5e service cinder] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:09:25.836760 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c864e1a-a08c-4bdc-9bea-26831713ab5e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:09:25.838363 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-8c864e1a-a08c-4bdc-9bea-26831713ab5e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:09:25.956037 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:09:25.973215 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c864e1a-a08c-4bdc-9bea-26831713ab5e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:09:25.975117 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c864e1a-a08c-4bdc-9bea-26831713ab5e service cinder] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:09:25.998406 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 688/1373] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:09:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:09:26.385016 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66eadcb5-88f8-4691-b501-74414ec4739f None None] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:09:26.385210 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66eadcb5-88f8-4691-b501-74414ec4739f None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:09:26.385483 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66eadcb5-88f8-4691-b501-74414ec4739f None None] PATH_INFO: `/` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:09:26.386826 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 686/1374] 10.222.0.62 () {54 vars in 927 bytes} [Wed Aug 24 06:09:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:09:26.392139 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883aad6a-4bac-46b9-902c-03378bcf9f70 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:09:26.392426 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883aad6a-4bac-46b9-902c-03378bcf9f70 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:09:26.392742 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883aad6a-4bac-46b9-902c-03378bcf9f70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:09:26.458604 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-883aad6a-4bac-46b9-902c-03378bcf9f70 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:09:26.459816 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-883aad6a-4bac-46b9-902c-03378bcf9f70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:09:26.556571 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 689/1375] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Aug 24 06:09:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 06:09:26.850045 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66eadcb5-88f8-4691-b501-74414ec4739f None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:09:26.851578 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-66eadcb5-88f8-4691-b501-74414ec4739f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:09:27.000406 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66eadcb5-88f8-4691-b501-74414ec4739f None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:09:27.058534 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:09:27.116934 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdf2662-d544-40e4-9106-37804a185c08 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb6d3743340241f98cacb9e0d063298c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:09:27.117569 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdf2662-d544-40e4-9106-37804a185c08 service nova] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:09:27.117860 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdf2662-d544-40e4-9106-37804a185c08 service nova] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:09:27.118189 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdf2662-d544-40e4-9106-37804a185c08 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:09:27.119798 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-8fdf2662-d544-40e4-9106-37804a185c08 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:09:27.211536 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fdf2662-d544-40e4-9106-37804a185c08 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:09:27.213200 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fdf2662-d544-40e4-9106-37804a185c08 service nova] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:09:27.231606 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 687/1376] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:09:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:09:29.369554 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebbf67b-4786-438e-8405-8c1b3cdd78c7 None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:09:29.370056 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebbf67b-4786-438e-8405-8c1b3cdd78c7 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:09:29.370056 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebbf67b-4786-438e-8405-8c1b3cdd78c7 None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:09:29.371374 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 690/1377] 10.222.0.62 () {54 vars in 927 bytes} [Wed Aug 24 06:09:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:09:29.377285 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f4e9ff-fda6-4ec9-a841-6fb49fd37ee4 None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:09:29.377728 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f4e9ff-fda6-4ec9-a841-6fb49fd37ee4 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:09:29.377825 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f4e9ff-fda6-4ec9-a841-6fb49fd37ee4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:09:29.444721 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-86f4e9ff-fda6-4ec9-a841-6fb49fd37ee4 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:09:29.445928 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-86f4e9ff-fda6-4ec9-a841-6fb49fd37ee4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:09:29.452572 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:09:29.548925 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 688/1378] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Aug 24 06:09:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:09:29.862820 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebbf67b-4786-438e-8405-8c1b3cdd78c7 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:09:29.864581 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-0ebbf67b-4786-438e-8405-8c1b3cdd78c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:09:29.937225 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:09:30.033520 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebbf67b-4786-438e-8405-8c1b3cdd78c7 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:09:30.158997 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6099e91a-7544-4f78-b8d3-8645a8ef63aa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb6d3743340241f98cacb9e0d063298c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:09:30.159529 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6099e91a-7544-4f78-b8d3-8645a8ef63aa service nova] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:09:30.159809 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6099e91a-7544-4f78-b8d3-8645a8ef63aa service nova] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:09:30.160084 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6099e91a-7544-4f78-b8d3-8645a8ef63aa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:09:30.161727 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-6099e91a-7544-4f78-b8d3-8645a8ef63aa service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:09:30.181682 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:09:30.257978 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6099e91a-7544-4f78-b8d3-8645a8ef63aa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:09:30.259981 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6099e91a-7544-4f78-b8d3-8645a8ef63aa service nova] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:09:30.280813 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 691/1379] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:09:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:09:32.492121 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15b1b1c-14ec-467e-ad5e-05c207f82849 None None] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:09:32.492733 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15b1b1c-14ec-467e-ad5e-05c207f82849 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:09:32.492733 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15b1b1c-14ec-467e-ad5e-05c207f82849 None None] PATH_INFO: `/` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:09:32.494155 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 689/1380] 10.222.0.62 () {54 vars in 927 bytes} [Wed Aug 24 06:09:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:09:32.499931 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7433aefe-756a-453d-8f23-42535beb9891 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:09:32.500348 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7433aefe-756a-453d-8f23-42535beb9891 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:09:32.500456 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7433aefe-756a-453d-8f23-42535beb9891 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:09:32.583288 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-7433aefe-756a-453d-8f23-42535beb9891 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:09:32.584584 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-7433aefe-756a-453d-8f23-42535beb9891 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:09:32.691937 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:09:32.698767 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 692/1381] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Aug 24 06:09:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:09:32.988828 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15b1b1c-14ec-467e-ad5e-05c207f82849 None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:09:32.990500 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-e15b1b1c-14ec-467e-ad5e-05c207f82849 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:09:33.135811 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:09:33.147183 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15b1b1c-14ec-467e-ad5e-05c207f82849 None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:09:33.272094 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bb66e1-ded0-4be7-9254-4e803580f363 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb6d3743340241f98cacb9e0d063298c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:09:33.272659 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bb66e1-ded0-4be7-9254-4e803580f363 service nova] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:09:33.273038 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bb66e1-ded0-4be7-9254-4e803580f363 service nova] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:09:33.273410 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bb66e1-ded0-4be7-9254-4e803580f363 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:09:33.275124 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-d8bb66e1-ded0-4be7-9254-4e803580f363 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:09:33.388396 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bb66e1-ded0-4be7-9254-4e803580f363 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:09:33.391548 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bb66e1-ded0-4be7-9254-4e803580f363 service nova] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:09:33.411245 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:09:33.420547 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 690/1382] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:09:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:09:35.506753 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5303535d-6655-4eab-a507-5770443811eb None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:09:35.507466 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5303535d-6655-4eab-a507-5770443811eb None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:09:35.507466 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5303535d-6655-4eab-a507-5770443811eb None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:09:35.508662 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 693/1383] 10.222.0.62 () {54 vars in 927 bytes} [Wed Aug 24 06:09:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:09:35.514140 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a88d914-c385-4bbc-8537-6cae57b73f5c None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:09:35.514591 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a88d914-c385-4bbc-8537-6cae57b73f5c None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:09:35.514707 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a88d914-c385-4bbc-8537-6cae57b73f5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:09:35.702139 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-1a88d914-c385-4bbc-8537-6cae57b73f5c None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:09:35.703345 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-1a88d914-c385-4bbc-8537-6cae57b73f5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:09:35.802766 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 691/1384] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Aug 24 06:09:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 290 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 06:09:36.092222 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5303535d-6655-4eab-a507-5770443811eb None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:09:36.093762 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-5303535d-6655-4eab-a507-5770443811eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:09:36.242510 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5303535d-6655-4eab-a507-5770443811eb None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:09:36.314978 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:09:36.362655 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d022ea-7fc0-4107-8edc-6d0519167206 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb6d3743340241f98cacb9e0d063298c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:09:36.363174 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d022ea-7fc0-4107-8edc-6d0519167206 service nova] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:09:36.363451 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d022ea-7fc0-4107-8edc-6d0519167206 service nova] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:09:36.363772 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d022ea-7fc0-4107-8edc-6d0519167206 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:09:36.365340 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-39d022ea-7fc0-4107-8edc-6d0519167206 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:09:36.463928 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d022ea-7fc0-4107-8edc-6d0519167206 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:09:36.465626 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d022ea-7fc0-4107-8edc-6d0519167206 service nova] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:09:36.483830 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 694/1385] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:09:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:09:38.389904 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dffbc9-142f-44d2-b51e-b01683f337f0 None None] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:09:38.390599 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dffbc9-142f-44d2-b51e-b01683f337f0 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:09:38.390599 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dffbc9-142f-44d2-b51e-b01683f337f0 None None] PATH_INFO: `/` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:09:38.391779 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 692/1386] 10.222.0.62 () {54 vars in 927 bytes} [Wed Aug 24 06:09:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 06:09:38.398555 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edb5438-14b8-40f6-a7f9-921712d46587 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:09:38.399000 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edb5438-14b8-40f6-a7f9-921712d46587 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:09:38.399107 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edb5438-14b8-40f6-a7f9-921712d46587 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:09:38.474260 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-4edb5438-14b8-40f6-a7f9-921712d46587 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:09:38.475497 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-4edb5438-14b8-40f6-a7f9-921712d46587 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:09:38.507924 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:09:38.579744 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 695/1387] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Aug 24 06:09:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:09:38.952313 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23dffbc9-142f-44d2-b51e-b01683f337f0 None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:09:38.954456 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-23dffbc9-142f-44d2-b51e-b01683f337f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:09:39.042791 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:09:39.112957 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23dffbc9-142f-44d2-b51e-b01683f337f0 None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:09:39.231980 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9761e02-c9d2-4564-a5b0-a8166d1e9815 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb6d3743340241f98cacb9e0d063298c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:09:39.232613 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9761e02-c9d2-4564-a5b0-a8166d1e9815 service nova] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:09:39.232894 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9761e02-c9d2-4564-a5b0-a8166d1e9815 service nova] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:09:39.233171 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9761e02-c9d2-4564-a5b0-a8166d1e9815 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:09:39.234796 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-e9761e02-c9d2-4564-a5b0-a8166d1e9815 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:09:39.281325 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:09:39.339159 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9761e02-c9d2-4564-a5b0-a8166d1e9815 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:09:39.341322 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9761e02-c9d2-4564-a5b0-a8166d1e9815 service nova] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:09:39.359664 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 693/1388] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:09:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:09:41.476861 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3362cd3f-068e-4252-8895-a7c7be9775f3 None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:09:41.477516 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3362cd3f-068e-4252-8895-a7c7be9775f3 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:09:41.477516 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3362cd3f-068e-4252-8895-a7c7be9775f3 None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:09:41.479247 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 696/1389] 10.222.0.62 () {54 vars in 927 bytes} [Wed Aug 24 06:09:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:09:41.484487 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa0da21-6c50-4a28-bdd1-456c659f256d None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:09:41.485058 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa0da21-6c50-4a28-bdd1-456c659f256d None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:09:41.485177 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa0da21-6c50-4a28-bdd1-456c659f256d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:09:41.564514 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-8fa0da21-6c50-4a28-bdd1-456c659f256d None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:09:41.565723 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-8fa0da21-6c50-4a28-bdd1-456c659f256d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:09:41.584249 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90e1d26-1f15-4e6f-a3a2-cac8dbebb231 None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:09:41.584249 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90e1d26-1f15-4e6f-a3a2-cac8dbebb231 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:09:41.584249 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90e1d26-1f15-4e6f-a3a2-cac8dbebb231 None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:09:41.584249 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 697/1390] 10.222.0.62 () {54 vars in 927 bytes} [Wed Aug 24 06:09:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 06:09:41.591069 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1591a9d-1509-493a-a1d4-dd3e5d7150aa None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:09:41.591352 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1591a9d-1509-493a-a1d4-dd3e5d7150aa None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:09:41.591627 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1591a9d-1509-493a-a1d4-dd3e5d7150aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:09:41.676102 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 694/1391] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Aug 24 06:09:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:09:41.676378 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-f1591a9d-1509-493a-a1d4-dd3e5d7150aa None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:09:41.677407 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-f1591a9d-1509-493a-a1d4-dd3e5d7150aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:09:41.791149 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 698/1392] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Aug 24 06:09:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 06:09:41.965552 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa0da21-6c50-4a28-bdd1-456c659f256d None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:09:41.969916 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-8fa0da21-6c50-4a28-bdd1-456c659f256d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:09:41.985169 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:09:42.095393 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15232b51-4653-405f-b9f4-975986c02f1e None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:09:42.095871 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15232b51-4653-405f-b9f4-975986c02f1e None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:09:42.096205 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15232b51-4653-405f-b9f4-975986c02f1e None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:09:42.097658 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 699/1393] 10.222.0.62 () {54 vars in 967 bytes} [Wed Aug 24 06:09:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 06:09:42.102216 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa0da21-6c50-4a28-bdd1-456c659f256d None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:09:42.102827 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed70e147-552d-4a9b-bc02-a186a086d483 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:09:42.103114 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed70e147-552d-4a9b-bc02-a186a086d483 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:09:42.103390 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed70e147-552d-4a9b-bc02-a186a086d483 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:09:42.125161 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:09:42.195598 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-ed70e147-552d-4a9b-bc02-a186a086d483 None None] MFA Rules not processed for user `c7d52162cbdc4218b577ecc1e69a88c9`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:09:42.196913 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-ed70e147-552d-4a9b-bc02-a186a086d483 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:09:42.246308 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513ea782-4c8d-4e25-bff9-baf6ec3b3dcc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb6d3743340241f98cacb9e0d063298c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:09:42.247615 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513ea782-4c8d-4e25-bff9-baf6ec3b3dcc service nova] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:09:42.247891 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513ea782-4c8d-4e25-bff9-baf6ec3b3dcc service nova] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:09:42.248156 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513ea782-4c8d-4e25-bff9-baf6ec3b3dcc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:09:42.249875 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-513ea782-4c8d-4e25-bff9-baf6ec3b3dcc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:09:42.259424 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:09:42.281992 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 700/1394] 10.222.0.62 () {58 vars in 1064 bytes} [Wed Aug 24 06:09:42 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 06:09:42.289908 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed70e147-552d-4a9b-bc02-a186a086d483 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:09:42.300988 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed70e147-552d-4a9b-bc02-a186a086d483 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:09:42.349479 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513ea782-4c8d-4e25-bff9-baf6ec3b3dcc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:09:42.351124 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513ea782-4c8d-4e25-bff9-baf6ec3b3dcc service nova] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:09:42.373352 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 695/1395] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:09:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:09:42.388384 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65af5ed5-2ad6-406f-aaea-bb8f3b38aaa5 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7d52162cbdc4218b577ecc1e69a88c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:09:42.389025 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65af5ed5-2ad6-406f-aaea-bb8f3b38aaa5 service glance] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:09:42.389304 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65af5ed5-2ad6-406f-aaea-bb8f3b38aaa5 service glance] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:09:42.389571 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65af5ed5-2ad6-406f-aaea-bb8f3b38aaa5 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:09:42.391200 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-65af5ed5-2ad6-406f-aaea-bb8f3b38aaa5 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:09:42.422560 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:09:42.510103 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65af5ed5-2ad6-406f-aaea-bb8f3b38aaa5 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:09:42.511632 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65af5ed5-2ad6-406f-aaea-bb8f3b38aaa5 service glance] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:09:42.535883 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 701/1396] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:09:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 246 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 06:09:43.430162 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644165b6-a21f-4b11-b75f-ea8e73f148a0 None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:09:43.430670 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644165b6-a21f-4b11-b75f-ea8e73f148a0 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:09:43.436839 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644165b6-a21f-4b11-b75f-ea8e73f148a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:09:43.521350 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-644165b6-a21f-4b11-b75f-ea8e73f148a0 None None] MFA Rules not processed for user `b82726a12436435cb931b986ef884c2e`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:09:43.522786 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-644165b6-a21f-4b11-b75f-ea8e73f148a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:09:43.625218 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:09:43.629013 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 696/1397] 10.222.0.62 () {58 vars in 1009 bytes} [Wed Aug 24 06:09:43 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:09:43.677226 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c94178-1da6-45fb-95fd-4fba29c3c912 None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:09:43.677780 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c94178-1da6-45fb-95fd-4fba29c3c912 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:09:43.677780 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c94178-1da6-45fb-95fd-4fba29c3c912 None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:09:43.679184 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 702/1398] 10.222.0.62 () {54 vars in 943 bytes} [Wed Aug 24 06:09:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:09:43.686866 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ced99d-47e7-4642-85e9-1a5174ce0d6b None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:09:43.687380 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ced99d-47e7-4642-85e9-1a5174ce0d6b None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:09:43.687869 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ced99d-47e7-4642-85e9-1a5174ce0d6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:09:43.782734 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-00ced99d-47e7-4642-85e9-1a5174ce0d6b None None] MFA Rules not processed for user `14d4fed04fb144d08918bd5d4f808d23`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:09:43.784008 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-00ced99d-47e7-4642-85e9-1a5174ce0d6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:09:43.904718 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 697/1399] 10.222.0.62 () {58 vars in 1040 bytes} [Wed Aug 24 06:09:43 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:09:43.914251 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c94178-1da6-45fb-95fd-4fba29c3c912 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:09:43.915757 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-a2c94178-1da6-45fb-95fd-4fba29c3c912 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:09:44.065852 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:09:44.068266 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c94178-1da6-45fb-95fd-4fba29c3c912 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:09:44.190628 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f2a6db-5bd8-4c6f-9164-c698eb79de99 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14d4fed04fb144d08918bd5d4f808d23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:09:44.191252 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f2a6db-5bd8-4c6f-9164-c698eb79de99 service swift] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:09:44.191526 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f2a6db-5bd8-4c6f-9164-c698eb79de99 service swift] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:09:44.191794 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f2a6db-5bd8-4c6f-9164-c698eb79de99 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:09:44.193507 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-52f2a6db-5bd8-4c6f-9164-c698eb79de99 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:09:44.288545 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f2a6db-5bd8-4c6f-9164-c698eb79de99 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:09:44.290527 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f2a6db-5bd8-4c6f-9164-c698eb79de99 service swift] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:09:44.299406 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 703/1400] 10.222.0.62 () {60 vars in 1384 bytes} [Wed Aug 24 06:09:43 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 06:09:44.485416 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb3b243-7eeb-417b-8854-1b93fccbda24 None None] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:09:44.486230 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb3b243-7eeb-417b-8854-1b93fccbda24 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:09:44.486393 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb3b243-7eeb-417b-8854-1b93fccbda24 None None] PATH_INFO: `/` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:09:44.487902 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 698/1401] 10.222.0.62 () {54 vars in 927 bytes} [Wed Aug 24 06:09:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:09:44.493448 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455c1a9d-e062-47e4-b405-2c9f934c1c2c None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:09:44.493730 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455c1a9d-e062-47e4-b405-2c9f934c1c2c None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:09:44.494008 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455c1a9d-e062-47e4-b405-2c9f934c1c2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:09:44.653962 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:09:44.687694 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-455c1a9d-e062-47e4-b405-2c9f934c1c2c None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:09:44.690412 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-455c1a9d-e062-47e4-b405-2c9f934c1c2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:09:44.794122 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 704/1402] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Aug 24 06:09:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 301 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:09:45.095020 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb3b243-7eeb-417b-8854-1b93fccbda24 None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:09:45.096269 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-deb3b243-7eeb-417b-8854-1b93fccbda24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:09:45.178797 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:09:45.301410 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb3b243-7eeb-417b-8854-1b93fccbda24 None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:09:45.447603 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9081c205-8bd0-4332-a0a2-e31ef41a608a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb6d3743340241f98cacb9e0d063298c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:09:45.448305 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9081c205-8bd0-4332-a0a2-e31ef41a608a service nova] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:09:45.448712 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9081c205-8bd0-4332-a0a2-e31ef41a608a service nova] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:09:45.448989 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9081c205-8bd0-4332-a0a2-e31ef41a608a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:09:45.450669 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-9081c205-8bd0-4332-a0a2-e31ef41a608a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:09:45.563353 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9081c205-8bd0-4332-a0a2-e31ef41a608a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:09:45.565066 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9081c205-8bd0-4332-a0a2-e31ef41a608a service nova] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:09:45.592825 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 699/1403] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:09:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 499 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:09:45.602388 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:09:47.753716 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591192ac-52f3-4781-9129-27561936ac9b None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:09:47.755062 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591192ac-52f3-4781-9129-27561936ac9b None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:09:47.755896 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591192ac-52f3-4781-9129-27561936ac9b None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:09:47.757903 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 705/1404] 10.222.0.62 () {54 vars in 927 bytes} [Wed Aug 24 06:09:47 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:09:47.764108 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b22b50-a22a-403c-bbc6-5c8424f18e56 None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:09:47.764642 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b22b50-a22a-403c-bbc6-5c8424f18e56 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:09:47.764735 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b22b50-a22a-403c-bbc6-5c8424f18e56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:09:47.852272 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-01b22b50-a22a-403c-bbc6-5c8424f18e56 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:09:47.853476 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-01b22b50-a22a-403c-bbc6-5c8424f18e56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:09:47.960284 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:09:47.972661 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 700/1405] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Aug 24 06:09:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:09:48.263755 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591192ac-52f3-4781-9129-27561936ac9b None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:09:48.264903 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-591192ac-52f3-4781-9129-27561936ac9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:09:48.384881 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:09:48.435571 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591192ac-52f3-4781-9129-27561936ac9b None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:09:48.598579 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048a6dac-12e2-4b3b-b607-0c52206b2584 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb6d3743340241f98cacb9e0d063298c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:09:48.599536 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048a6dac-12e2-4b3b-b607-0c52206b2584 service nova] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:09:48.599885 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048a6dac-12e2-4b3b-b607-0c52206b2584 service nova] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:09:48.600213 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048a6dac-12e2-4b3b-b607-0c52206b2584 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:09:48.603559 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-048a6dac-12e2-4b3b-b607-0c52206b2584 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:09:48.700732 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:09:48.729926 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048a6dac-12e2-4b3b-b607-0c52206b2584 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:09:48.732229 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048a6dac-12e2-4b3b-b607-0c52206b2584 service nova] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:09:48.753458 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 706/1406] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:09:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 492 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:09:50.894694 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c00672-158d-428f-acee-65084334d279 None None] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:09:50.894694 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c00672-158d-428f-acee-65084334d279 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:09:50.894694 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c00672-158d-428f-acee-65084334d279 None None] PATH_INFO: `/` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:09:50.894694 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 701/1407] 10.222.0.62 () {54 vars in 927 bytes} [Wed Aug 24 06:09:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:09:50.901042 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2035889f-f715-46d7-87b9-380342e409bb None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:09:50.901519 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2035889f-f715-46d7-87b9-380342e409bb None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:09:50.901608 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2035889f-f715-46d7-87b9-380342e409bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:09:50.973279 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-2035889f-f715-46d7-87b9-380342e409bb None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:09:50.974395 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-2035889f-f715-46d7-87b9-380342e409bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:09:51.084894 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 707/1408] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Aug 24 06:09:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:09:51.390451 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c00672-158d-428f-acee-65084334d279 None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:09:51.392079 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-b3c00672-158d-428f-acee-65084334d279 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:09:51.597409 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c00672-158d-428f-acee-65084334d279 None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:09:51.707771 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:09:51.774516 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986b7dd8-59e9-4298-98a2-b3c985fe9e29 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb6d3743340241f98cacb9e0d063298c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:09:51.774516 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986b7dd8-59e9-4298-98a2-b3c985fe9e29 service nova] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:09:51.774516 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986b7dd8-59e9-4298-98a2-b3c985fe9e29 service nova] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:09:51.774985 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986b7dd8-59e9-4298-98a2-b3c985fe9e29 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:09:51.776696 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-986b7dd8-59e9-4298-98a2-b3c985fe9e29 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:09:51.894174 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986b7dd8-59e9-4298-98a2-b3c985fe9e29 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:09:51.900704 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986b7dd8-59e9-4298-98a2-b3c985fe9e29 service nova] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:09:51.927050 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 702/1409] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:09:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 537 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:10:01.082889 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=79 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:10:01.924278 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=64 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:10:08.660447 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46073e06-1e5c-4aa0-ab19-a7f04ae31994 None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:10:08.661002 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46073e06-1e5c-4aa0-ab19-a7f04ae31994 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:10:08.661002 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46073e06-1e5c-4aa0-ab19-a7f04ae31994 None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:10:08.662319 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 708/1410] 10.222.0.62 () {54 vars in 918 bytes} [Wed Aug 24 06:10:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:10:08.667656 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b3e63c-0db9-4488-a520-cc4d193bcec5 None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:10:08.668167 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b3e63c-0db9-4488-a520-cc4d193bcec5 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:10:08.668264 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b3e63c-0db9-4488-a520-cc4d193bcec5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:10:08.749122 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-f2b3e63c-0db9-4488-a520-cc4d193bcec5 None None] MFA Rules not processed for user `abe9d8c9291845dfa4ef55034e5c9e32`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:10:08.750349 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-f2b3e63c-0db9-4488-a520-cc4d193bcec5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:10:08.856180 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 703/1411] 10.222.0.62 () {58 vars in 1015 bytes} [Wed Aug 24 06:10:08 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:10:08.870912 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c628b29-96a9-4ae5-a97b-a97f73c1cec0 None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:10:08.871193 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c628b29-96a9-4ae5-a97b-a97f73c1cec0 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:10:08.871462 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c628b29-96a9-4ae5-a97b-a97f73c1cec0 None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:10:08.872898 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 709/1412] 10.222.0.62 () {54 vars in 918 bytes} [Wed Aug 24 06:10:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:10:08.877830 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b7d95d-977a-4387-b0c5-3f1d8bdd8851 None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:10:08.878105 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b7d95d-977a-4387-b0c5-3f1d8bdd8851 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:10:08.878383 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b7d95d-977a-4387-b0c5-3f1d8bdd8851 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:10:08.952182 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:10:08.954765 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-11b7d95d-977a-4387-b0c5-3f1d8bdd8851 None None] MFA Rules not processed for user `fb6d3743340241f98cacb9e0d063298c`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:10:08.955969 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-11b7d95d-977a-4387-b0c5-3f1d8bdd8851 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:10:09.075927 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 704/1413] 10.222.0.62 () {58 vars in 1015 bytes} [Wed Aug 24 06:10:08 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:10:11.472335 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbafc4d8-e963-48fb-965f-ddaf35da2853 None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:10:11.472931 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbafc4d8-e963-48fb-965f-ddaf35da2853 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:10:11.472931 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbafc4d8-e963-48fb-965f-ddaf35da2853 None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:10:11.474139 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 710/1414] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 06:10:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 06:10:11.480076 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b806ea-52e2-4415-8aee-695c60e64ded None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:10:11.480541 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b806ea-52e2-4415-8aee-695c60e64ded None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:10:11.480634 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b806ea-52e2-4415-8aee-695c60e64ded None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:10:11.544805 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-f9b806ea-52e2-4415-8aee-695c60e64ded None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:10:11.545986 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-f9b806ea-52e2-4415-8aee-695c60e64ded None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:10:11.598950 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:10:11.640696 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 705/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 06:10:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 06:10:11.928465 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbafc4d8-e963-48fb-965f-ddaf35da2853 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:10:11.930030 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-fbafc4d8-e963-48fb-965f-ddaf35da2853 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:10:12.080115 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbafc4d8-e963-48fb-965f-ddaf35da2853 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:10:12.156904 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:10:12.199907 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51089482-32a7-49c6-b1d3-7fc19d86672b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb6d3743340241f98cacb9e0d063298c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:10:12.200452 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51089482-32a7-49c6-b1d3-7fc19d86672b service nova] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:10:12.200747 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51089482-32a7-49c6-b1d3-7fc19d86672b service nova] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:10:12.201012 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51089482-32a7-49c6-b1d3-7fc19d86672b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:10:12.202761 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-51089482-32a7-49c6-b1d3-7fc19d86672b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:10:12.292820 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51089482-32a7-49c6-b1d3-7fc19d86672b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:10:12.294422 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51089482-32a7-49c6-b1d3-7fc19d86672b service nova] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:10:12.312759 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 711/1416] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:10:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:10:13.211875 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc882c0-0fa5-49f8-9ff1-3c1862388ac0 None None] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:10:13.212432 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc882c0-0fa5-49f8-9ff1-3c1862388ac0 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:10:13.212432 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc882c0-0fa5-49f8-9ff1-3c1862388ac0 None None] PATH_INFO: `/` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:10:13.214300 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 706/1417] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 06:10:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:10:13.219353 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70293e89-0839-48a5-94fa-ba35747c033b None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:10:13.219774 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70293e89-0839-48a5-94fa-ba35747c033b None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:10:13.219891 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70293e89-0839-48a5-94fa-ba35747c033b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:10:13.299592 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-70293e89-0839-48a5-94fa-ba35747c033b None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:10:13.301122 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-70293e89-0839-48a5-94fa-ba35747c033b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:10:13.330191 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:10:13.403500 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 712/1418] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 06:10:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:10:13.555541 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc882c0-0fa5-49f8-9ff1-3c1862388ac0 None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:10:13.558134 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-ecc882c0-0fa5-49f8-9ff1-3c1862388ac0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:10:13.707964 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc882c0-0fa5-49f8-9ff1-3c1862388ac0 None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:10:13.747915 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:10:13.819899 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32893972-ca6b-4997-8e5c-2c44f92752ee service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb6d3743340241f98cacb9e0d063298c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:10:13.820427 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32893972-ca6b-4997-8e5c-2c44f92752ee service nova] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:10:13.820739 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32893972-ca6b-4997-8e5c-2c44f92752ee service nova] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:10:13.821016 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32893972-ca6b-4997-8e5c-2c44f92752ee service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:10:13.822617 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-32893972-ca6b-4997-8e5c-2c44f92752ee service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:10:13.912678 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32893972-ca6b-4997-8e5c-2c44f92752ee service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:10:13.914282 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32893972-ca6b-4997-8e5c-2c44f92752ee service nova] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:10:13.932175 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 707/1419] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:10:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:10:14.810060 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ac87cf-9576-462b-943f-c801c0bfb2a7 None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:10:14.810636 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ac87cf-9576-462b-943f-c801c0bfb2a7 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:10:14.810736 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ac87cf-9576-462b-943f-c801c0bfb2a7 None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:10:14.812069 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 713/1420] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 06:10:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:10:14.816868 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59de19d2-5176-419c-be2f-5547c323bc30 None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:10:14.817305 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59de19d2-5176-419c-be2f-5547c323bc30 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:10:14.817409 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59de19d2-5176-419c-be2f-5547c323bc30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:10:14.870712 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:10:14.891322 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-59de19d2-5176-419c-be2f-5547c323bc30 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:10:14.892523 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-59de19d2-5176-419c-be2f-5547c323bc30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:10:14.995082 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 708/1421] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 06:10:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:10:15.147774 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ac87cf-9576-462b-943f-c801c0bfb2a7 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:10:15.149295 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-d9ac87cf-9576-462b-943f-c801c0bfb2a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:10:15.293558 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ac87cf-9576-462b-943f-c801c0bfb2a7 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:10:15.305577 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:10:15.403454 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec68e83-04a0-41ba-8ce1-deaa7d798ba1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb6d3743340241f98cacb9e0d063298c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:10:15.404055 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec68e83-04a0-41ba-8ce1-deaa7d798ba1 service nova] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:10:15.404332 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec68e83-04a0-41ba-8ce1-deaa7d798ba1 service nova] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:10:15.404645 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec68e83-04a0-41ba-8ce1-deaa7d798ba1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:10:15.406220 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-dec68e83-04a0-41ba-8ce1-deaa7d798ba1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:10:15.492150 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec68e83-04a0-41ba-8ce1-deaa7d798ba1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:10:15.493774 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec68e83-04a0-41ba-8ce1-deaa7d798ba1 service nova] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:10:15.512285 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 714/1422] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:10:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:10:16.379154 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c473c3-7606-493d-97d7-70aef2b1d57c None None] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:10:16.379731 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c473c3-7606-493d-97d7-70aef2b1d57c None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:10:16.379731 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c473c3-7606-493d-97d7-70aef2b1d57c None None] PATH_INFO: `/` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:10:16.381001 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 709/1423] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 06:10:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:10:16.385949 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07939c08-cdae-4428-95cb-02e7f613237e None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:10:16.386367 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07939c08-cdae-4428-95cb-02e7f613237e None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:10:16.386516 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07939c08-cdae-4428-95cb-02e7f613237e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:10:16.406154 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:10:16.462691 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-07939c08-cdae-4428-95cb-02e7f613237e None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:10:16.464406 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-07939c08-cdae-4428-95cb-02e7f613237e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:10:16.563649 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 715/1424] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 06:10:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 06:10:16.718557 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c473c3-7606-493d-97d7-70aef2b1d57c None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:10:16.720850 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-c7c473c3-7606-493d-97d7-70aef2b1d57c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:10:16.837067 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:10:16.876085 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c473c3-7606-493d-97d7-70aef2b1d57c None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:10:16.989350 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab329ef-35b7-4aa1-811c-fdc0192057e6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb6d3743340241f98cacb9e0d063298c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:10:16.989866 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab329ef-35b7-4aa1-811c-fdc0192057e6 service nova] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:10:16.990135 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab329ef-35b7-4aa1-811c-fdc0192057e6 service nova] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:10:16.990401 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab329ef-35b7-4aa1-811c-fdc0192057e6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:10:16.991992 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-3ab329ef-35b7-4aa1-811c-fdc0192057e6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:10:17.072438 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:10:17.086574 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab329ef-35b7-4aa1-811c-fdc0192057e6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:10:17.088288 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab329ef-35b7-4aa1-811c-fdc0192057e6 service nova] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:10:17.109076 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 710/1425] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:10:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:10:17.970670 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d79d040-13c2-4feb-8ab3-39566c6cd5c7 None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:10:17.971364 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d79d040-13c2-4feb-8ab3-39566c6cd5c7 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:10:17.971364 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d79d040-13c2-4feb-8ab3-39566c6cd5c7 None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:10:17.972580 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 716/1426] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 06:10:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:10:17.977637 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec81cf26-1658-4065-a150-94b652b6d37f None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:10:17.978089 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec81cf26-1658-4065-a150-94b652b6d37f None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:10:17.978200 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec81cf26-1658-4065-a150-94b652b6d37f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:10:18.053284 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-ec81cf26-1658-4065-a150-94b652b6d37f None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:10:18.054483 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-ec81cf26-1658-4065-a150-94b652b6d37f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:10:18.151038 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 711/1427] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 06:10:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:10:18.302882 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d79d040-13c2-4feb-8ab3-39566c6cd5c7 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:10:18.304380 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-7d79d040-13c2-4feb-8ab3-39566c6cd5c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:10:18.388415 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:10:18.449356 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d79d040-13c2-4feb-8ab3-39566c6cd5c7 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:10:18.559999 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad36af7-8068-4ff8-ac18-98c4531a0e0a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb6d3743340241f98cacb9e0d063298c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:10:18.560545 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad36af7-8068-4ff8-ac18-98c4531a0e0a service nova] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:10:18.560823 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad36af7-8068-4ff8-ac18-98c4531a0e0a service nova] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:10:18.561090 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad36af7-8068-4ff8-ac18-98c4531a0e0a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:10:18.562658 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-0ad36af7-8068-4ff8-ac18-98c4531a0e0a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:10:18.610537 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:10:18.649480 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad36af7-8068-4ff8-ac18-98c4531a0e0a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:10:18.651563 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad36af7-8068-4ff8-ac18-98c4531a0e0a service nova] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:10:18.669724 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 717/1428] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:10:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:10:19.571267 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07267fe7-20c1-4960-b7e7-ac7c958afc46 None None] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:10:19.571725 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07267fe7-20c1-4960-b7e7-ac7c958afc46 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:10:19.571827 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07267fe7-20c1-4960-b7e7-ac7c958afc46 None None] PATH_INFO: `/` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:10:19.573224 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 712/1429] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 06:10:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:10:19.578327 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d774d98-6b13-4c30-99f8-1d7bbead8777 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:10:19.578777 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d774d98-6b13-4c30-99f8-1d7bbead8777 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:10:19.578893 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d774d98-6b13-4c30-99f8-1d7bbead8777 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:10:19.665807 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-8d774d98-6b13-4c30-99f8-1d7bbead8777 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:10:19.667571 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-8d774d98-6b13-4c30-99f8-1d7bbead8777 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:10:19.768441 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 718/1430] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 06:10:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:10:19.924136 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07267fe7-20c1-4960-b7e7-ac7c958afc46 None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:10:19.925944 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-07267fe7-20c1-4960-b7e7-ac7c958afc46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:10:19.982583 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:10:20.084929 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07267fe7-20c1-4960-b7e7-ac7c958afc46 None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:10:20.197307 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c434dad5-50df-4401-9374-297105196d58 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb6d3743340241f98cacb9e0d063298c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:10:20.197940 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c434dad5-50df-4401-9374-297105196d58 service nova] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:10:20.198221 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c434dad5-50df-4401-9374-297105196d58 service nova] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:10:20.198495 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c434dad5-50df-4401-9374-297105196d58 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:10:20.200080 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-c434dad5-50df-4401-9374-297105196d58 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:10:20.206954 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:10:20.290444 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c434dad5-50df-4401-9374-297105196d58 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:10:20.292042 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c434dad5-50df-4401-9374-297105196d58 service nova] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:10:20.310145 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 713/1431] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:10:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:10:21.179547 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe5bde2-b000-48bf-b4d5-ce786b2446e5 None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:10:21.180179 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe5bde2-b000-48bf-b4d5-ce786b2446e5 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:10:21.180179 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe5bde2-b000-48bf-b4d5-ce786b2446e5 None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:10:21.181478 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 719/1432] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 06:10:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:10:21.187799 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f036d7f7-b19a-438b-b361-fd566bebc5ea None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:10:21.188320 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f036d7f7-b19a-438b-b361-fd566bebc5ea None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:10:21.188320 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f036d7f7-b19a-438b-b361-fd566bebc5ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:10:21.268638 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-f036d7f7-b19a-438b-b361-fd566bebc5ea None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:10:21.269860 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-f036d7f7-b19a-438b-b361-fd566bebc5ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:10:21.342165 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:10:21.372792 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 714/1433] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 06:10:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:10:21.522574 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe5bde2-b000-48bf-b4d5-ce786b2446e5 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:10:21.524033 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-7fe5bde2-b000-48bf-b4d5-ce786b2446e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:10:21.544039 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:10:21.679410 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe5bde2-b000-48bf-b4d5-ce786b2446e5 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:10:21.769244 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:10:21.795645 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d74133-9562-4dad-93f9-6d38d4075379 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb6d3743340241f98cacb9e0d063298c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:10:21.796177 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d74133-9562-4dad-93f9-6d38d4075379 service nova] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:10:21.796452 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d74133-9562-4dad-93f9-6d38d4075379 service nova] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:10:21.796809 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d74133-9562-4dad-93f9-6d38d4075379 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:10:21.799104 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-46d74133-9562-4dad-93f9-6d38d4075379 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:10:21.889303 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d74133-9562-4dad-93f9-6d38d4075379 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:10:21.890926 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d74133-9562-4dad-93f9-6d38d4075379 service nova] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:10:21.909959 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 720/1434] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:10:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 06:10:22.782407 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54aaf1f8-59ac-4cc1-8c7b-869ab1071972 None None] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:10:22.783035 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54aaf1f8-59ac-4cc1-8c7b-869ab1071972 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:10:22.783035 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54aaf1f8-59ac-4cc1-8c7b-869ab1071972 None None] PATH_INFO: `/` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:10:22.784188 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 715/1435] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 06:10:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:10:22.789395 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab21a75c-30fb-4aee-ba92-110531b9f903 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:10:22.789841 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab21a75c-30fb-4aee-ba92-110531b9f903 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:10:22.789966 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab21a75c-30fb-4aee-ba92-110531b9f903 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:10:22.874840 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-ab21a75c-30fb-4aee-ba92-110531b9f903 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:10:22.876051 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-ab21a75c-30fb-4aee-ba92-110531b9f903 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:10:22.919029 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:10:22.975032 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 721/1436] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 06:10:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:10:23.126518 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54aaf1f8-59ac-4cc1-8c7b-869ab1071972 None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:10:23.127970 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-54aaf1f8-59ac-4cc1-8c7b-869ab1071972 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:10:23.276980 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54aaf1f8-59ac-4cc1-8c7b-869ab1071972 None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:10:23.331867 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:10:23.393560 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa571e5a-da1a-4376-a94e-d22080c1c5ce service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb6d3743340241f98cacb9e0d063298c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:10:23.394137 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa571e5a-da1a-4376-a94e-d22080c1c5ce service nova] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:10:23.394453 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa571e5a-da1a-4376-a94e-d22080c1c5ce service nova] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:10:23.394729 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa571e5a-da1a-4376-a94e-d22080c1c5ce service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:10:23.396470 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-aa571e5a-da1a-4376-a94e-d22080c1c5ce service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:10:23.486241 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa571e5a-da1a-4376-a94e-d22080c1c5ce service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:10:23.487859 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa571e5a-da1a-4376-a94e-d22080c1c5ce service nova] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:10:23.505215 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 716/1437] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:10:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:10:24.382190 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c881a28f-79e1-4271-b916-6b2cc5c2df8e None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:10:24.382759 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c881a28f-79e1-4271-b916-6b2cc5c2df8e None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:10:24.382759 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c881a28f-79e1-4271-b916-6b2cc5c2df8e None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:10:24.384063 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 722/1438] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 06:10:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:10:24.389111 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e93d424-36c9-43ec-97a2-0f00be369dc7 None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:10:24.389552 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e93d424-36c9-43ec-97a2-0f00be369dc7 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:10:24.389660 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e93d424-36c9-43ec-97a2-0f00be369dc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:10:24.465554 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-7e93d424-36c9-43ec-97a2-0f00be369dc7 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:10:24.466779 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-7e93d424-36c9-43ec-97a2-0f00be369dc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:10:24.468948 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:10:24.570802 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 717/1439] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 06:10:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:10:24.722106 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c881a28f-79e1-4271-b916-6b2cc5c2df8e None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:10:24.723618 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-c881a28f-79e1-4271-b916-6b2cc5c2df8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:10:24.873263 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c881a28f-79e1-4271-b916-6b2cc5c2df8e None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:10:24.900183 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:10:24.987706 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdca47f-2e74-421c-b0ac-c606d11591c2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb6d3743340241f98cacb9e0d063298c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:10:24.988237 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdca47f-2e74-421c-b0ac-c606d11591c2 service nova] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:10:24.988540 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdca47f-2e74-421c-b0ac-c606d11591c2 service nova] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:10:24.988818 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdca47f-2e74-421c-b0ac-c606d11591c2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:10:24.990422 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-5cdca47f-2e74-421c-b0ac-c606d11591c2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:10:25.078851 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdca47f-2e74-421c-b0ac-c606d11591c2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:10:25.080582 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdca47f-2e74-421c-b0ac-c606d11591c2 service nova] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:10:25.099043 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 723/1440] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:10:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:10:26.955551 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ab04ee-e34e-47af-a433-9a3780305b73 None None] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:10:26.956094 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ab04ee-e34e-47af-a433-9a3780305b73 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:10:26.956094 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ab04ee-e34e-47af-a433-9a3780305b73 None None] PATH_INFO: `/` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:10:26.957544 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 718/1441] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 06:10:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:10:26.962968 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6191334-815b-4057-8679-490ef6703a8b None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:10:26.963429 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6191334-815b-4057-8679-490ef6703a8b None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:10:26.963529 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6191334-815b-4057-8679-490ef6703a8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:10:27.005297 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:10:27.033564 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-e6191334-815b-4057-8679-490ef6703a8b None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:10:27.034810 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-e6191334-815b-4057-8679-490ef6703a8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:10:27.133097 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 724/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 06:10:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:10:27.419155 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ab04ee-e34e-47af-a433-9a3780305b73 None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:10:27.420746 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-24ab04ee-e34e-47af-a433-9a3780305b73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:10:27.534804 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ab04ee-e34e-47af-a433-9a3780305b73 None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:10:27.568280 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:10:27.613331 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07e8206-01e9-4f60-8e51-91f0b2f3b6b9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bb6c79db75e4d169331c02133d1ec32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:10:27.613935 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07e8206-01e9-4f60-8e51-91f0b2f3b6b9 service neutron] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:10:27.614210 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07e8206-01e9-4f60-8e51-91f0b2f3b6b9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:10:27.614478 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07e8206-01e9-4f60-8e51-91f0b2f3b6b9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:10:27.616063 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-c07e8206-01e9-4f60-8e51-91f0b2f3b6b9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:10:27.726018 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07e8206-01e9-4f60-8e51-91f0b2f3b6b9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:10:27.727457 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07e8206-01e9-4f60-8e51-91f0b2f3b6b9 service neutron] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:10:27.746157 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 719/1443] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:10:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:10:29.814150 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2829e55-d7c1-48f4-bd76-c2832827a9de None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:10:29.814694 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2829e55-d7c1-48f4-bd76-c2832827a9de None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:10:29.814694 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2829e55-d7c1-48f4-bd76-c2832827a9de None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:10:29.815963 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 725/1444] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 06:10:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:10:29.821317 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f607be-95b8-4e63-b526-67cdc3300fe4 None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:10:29.821755 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f607be-95b8-4e63-b526-67cdc3300fe4 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:10:29.821878 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f607be-95b8-4e63-b526-67cdc3300fe4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:10:29.888818 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-70f607be-95b8-4e63-b526-67cdc3300fe4 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:10:29.892673 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-70f607be-95b8-4e63-b526-67cdc3300fe4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:10:29.903608 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:10:29.990713 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 720/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 06:10:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:10:30.276952 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2829e55-d7c1-48f4-bd76-c2832827a9de None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:10:30.278497 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-a2829e55-d7c1-48f4-bd76-c2832827a9de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:10:30.381931 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2829e55-d7c1-48f4-bd76-c2832827a9de None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:10:30.394519 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:10:30.456021 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32980690-2a43-4217-9b96-b579aa55852e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bb6c79db75e4d169331c02133d1ec32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:10:30.456656 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32980690-2a43-4217-9b96-b579aa55852e service neutron] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:10:30.456933 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32980690-2a43-4217-9b96-b579aa55852e service neutron] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:10:30.457198 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32980690-2a43-4217-9b96-b579aa55852e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:10:30.458802 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-32980690-2a43-4217-9b96-b579aa55852e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:10:30.565652 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32980690-2a43-4217-9b96-b579aa55852e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:10:30.567002 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32980690-2a43-4217-9b96-b579aa55852e service neutron] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:10:30.585064 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 726/1446] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:10:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:10:32.902857 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0556168-a140-41e4-867b-af3ddc831d53 None None] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:10:32.903400 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0556168-a140-41e4-867b-af3ddc831d53 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:10:32.903400 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0556168-a140-41e4-867b-af3ddc831d53 None None] PATH_INFO: `/` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:10:32.904792 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 721/1447] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 06:10:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:10:32.910416 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19559755-6796-4593-b6e4-913a87efec2d None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:10:32.910835 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19559755-6796-4593-b6e4-913a87efec2d None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:10:32.910971 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19559755-6796-4593-b6e4-913a87efec2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:10:32.960273 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:10:32.977542 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-19559755-6796-4593-b6e4-913a87efec2d None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:10:32.978818 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-19559755-6796-4593-b6e4-913a87efec2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:10:33.075544 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 727/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 06:10:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:10:33.359197 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0556168-a140-41e4-867b-af3ddc831d53 None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:10:33.360771 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-b0556168-a140-41e4-867b-af3ddc831d53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:10:33.468070 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0556168-a140-41e4-867b-af3ddc831d53 None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:10:33.508909 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:10:33.542819 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b113d9-00b0-476c-84b3-de45dbe0fe01 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bb6c79db75e4d169331c02133d1ec32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:10:33.543457 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b113d9-00b0-476c-84b3-de45dbe0fe01 service neutron] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:10:33.543736 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b113d9-00b0-476c-84b3-de45dbe0fe01 service neutron] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:10:33.544008 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b113d9-00b0-476c-84b3-de45dbe0fe01 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:10:33.546376 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-c3b113d9-00b0-476c-84b3-de45dbe0fe01 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:10:33.657005 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b113d9-00b0-476c-84b3-de45dbe0fe01 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:10:33.658392 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b113d9-00b0-476c-84b3-de45dbe0fe01 service neutron] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:10:33.676892 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 722/1449] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:10:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:10:35.713897 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557215c5-60e7-4d05-a7a4-3b366e3a2928 None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:10:35.714352 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557215c5-60e7-4d05-a7a4-3b366e3a2928 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:10:35.714465 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557215c5-60e7-4d05-a7a4-3b366e3a2928 None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:10:35.715806 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 728/1450] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 06:10:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:10:35.721382 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ea7cee-7175-4a69-bfa7-0d766f4e23f1 None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:10:35.721807 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ea7cee-7175-4a69-bfa7-0d766f4e23f1 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:10:35.721932 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ea7cee-7175-4a69-bfa7-0d766f4e23f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:10:35.790791 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-73ea7cee-7175-4a69-bfa7-0d766f4e23f1 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:10:35.792682 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-73ea7cee-7175-4a69-bfa7-0d766f4e23f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:10:35.814279 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:10:35.894067 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 723/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 06:10:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:10:36.183259 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557215c5-60e7-4d05-a7a4-3b366e3a2928 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:10:36.184835 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-557215c5-60e7-4d05-a7a4-3b366e3a2928 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:10:36.289600 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557215c5-60e7-4d05-a7a4-3b366e3a2928 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:10:36.310581 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:10:36.363800 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27bc9a8-16a3-46f4-b203-9e2de126bbd6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bb6c79db75e4d169331c02133d1ec32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:10:36.364424 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27bc9a8-16a3-46f4-b203-9e2de126bbd6 service neutron] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:10:36.364822 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27bc9a8-16a3-46f4-b203-9e2de126bbd6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:10:36.365097 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27bc9a8-16a3-46f4-b203-9e2de126bbd6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:10:36.366761 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-f27bc9a8-16a3-46f4-b203-9e2de126bbd6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:10:36.480390 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27bc9a8-16a3-46f4-b203-9e2de126bbd6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:10:36.481770 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27bc9a8-16a3-46f4-b203-9e2de126bbd6 service neutron] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:10:36.500720 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 729/1452] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:10:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:10:38.499948 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a13f6e-c10d-4b15-bc27-3bef19af759c None None] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:10:38.500561 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a13f6e-c10d-4b15-bc27-3bef19af759c None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:10:38.500561 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a13f6e-c10d-4b15-bc27-3bef19af759c None None] PATH_INFO: `/` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:10:38.501863 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 724/1453] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 06:10:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:10:38.507283 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027a50b5-5391-4a57-8898-f86db5c3cf4a None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:10:38.507738 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027a50b5-5391-4a57-8898-f86db5c3cf4a None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:10:38.507857 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027a50b5-5391-4a57-8898-f86db5c3cf4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:10:38.571437 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:10:38.574687 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-027a50b5-5391-4a57-8898-f86db5c3cf4a None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:10:38.575894 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-027a50b5-5391-4a57-8898-f86db5c3cf4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:10:38.672736 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 730/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 06:10:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:10:38.958560 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a13f6e-c10d-4b15-bc27-3bef19af759c None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:10:38.960119 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-c3a13f6e-c10d-4b15-bc27-3bef19af759c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:10:39.066798 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a13f6e-c10d-4b15-bc27-3bef19af759c None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:10:39.115210 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:10:39.140154 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9a2c0f-bbb1-4225-be80-b4dec223709c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bb6c79db75e4d169331c02133d1ec32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:10:39.140889 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9a2c0f-bbb1-4225-be80-b4dec223709c service neutron] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:10:39.141232 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9a2c0f-bbb1-4225-be80-b4dec223709c service neutron] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:10:39.141588 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9a2c0f-bbb1-4225-be80-b4dec223709c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:10:39.143304 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-3c9a2c0f-bbb1-4225-be80-b4dec223709c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:10:39.253407 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9a2c0f-bbb1-4225-be80-b4dec223709c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:10:39.254745 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9a2c0f-bbb1-4225-be80-b4dec223709c service neutron] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:10:39.273446 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 725/1455] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:10:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:10:41.347354 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24601bea-1a3f-4eef-9bcc-a735f5238d45 None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:10:41.347900 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24601bea-1a3f-4eef-9bcc-a735f5238d45 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:10:41.347900 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24601bea-1a3f-4eef-9bcc-a735f5238d45 None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:10:41.349365 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 731/1456] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 06:10:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:10:41.355043 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afe2515-5969-46ab-b888-e4a376fc7b03 None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:10:41.355473 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afe2515-5969-46ab-b888-e4a376fc7b03 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:10:41.355574 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afe2515-5969-46ab-b888-e4a376fc7b03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:10:41.422064 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-2afe2515-5969-46ab-b888-e4a376fc7b03 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:10:41.423281 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-2afe2515-5969-46ab-b888-e4a376fc7b03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:10:41.454317 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:10:41.529716 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 726/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 06:10:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:10:41.812835 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24601bea-1a3f-4eef-9bcc-a735f5238d45 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:10:41.814415 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-24601bea-1a3f-4eef-9bcc-a735f5238d45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:10:41.924907 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24601bea-1a3f-4eef-9bcc-a735f5238d45 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:10:41.957686 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:10:42.004099 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebfa1ea-1a7e-43f2-8c46-43e5b34e1244 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bb6c79db75e4d169331c02133d1ec32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:10:42.004774 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebfa1ea-1a7e-43f2-8c46-43e5b34e1244 service neutron] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:10:42.005079 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebfa1ea-1a7e-43f2-8c46-43e5b34e1244 service neutron] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:10:42.005351 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebfa1ea-1a7e-43f2-8c46-43e5b34e1244 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:10:42.006947 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-9ebfa1ea-1a7e-43f2-8c46-43e5b34e1244 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:10:42.119254 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebfa1ea-1a7e-43f2-8c46-43e5b34e1244 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:10:42.120807 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebfa1ea-1a7e-43f2-8c46-43e5b34e1244 service neutron] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:10:42.139225 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 732/1458] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:10:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:10:44.187009 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925cb518-0237-42c8-8a54-a8ff7248afe6 None None] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:10:44.187625 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925cb518-0237-42c8-8a54-a8ff7248afe6 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:10:44.187625 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925cb518-0237-42c8-8a54-a8ff7248afe6 None None] PATH_INFO: `/` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:10:44.188997 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 727/1459] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 06:10:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:10:44.194335 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e220bf15-c461-4b43-ad20-cdaa8e4194b9 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:10:44.194776 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e220bf15-c461-4b43-ad20-cdaa8e4194b9 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:10:44.200164 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e220bf15-c461-4b43-ad20-cdaa8e4194b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:10:44.262674 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-e220bf15-c461-4b43-ad20-cdaa8e4194b9 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:10:44.263886 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-e220bf15-c461-4b43-ad20-cdaa8e4194b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:10:44.272147 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:10:44.360102 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 733/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 06:10:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:10:44.649908 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925cb518-0237-42c8-8a54-a8ff7248afe6 None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:10:44.651477 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-925cb518-0237-42c8-8a54-a8ff7248afe6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:10:44.757438 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925cb518-0237-42c8-8a54-a8ff7248afe6 None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:10:44.813570 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:10:44.832787 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e1367d-d14c-4c3f-ae96-3245eefe96ee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bb6c79db75e4d169331c02133d1ec32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:10:44.833488 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e1367d-d14c-4c3f-ae96-3245eefe96ee service neutron] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:10:44.833765 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e1367d-d14c-4c3f-ae96-3245eefe96ee service neutron] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:10:44.834036 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e1367d-d14c-4c3f-ae96-3245eefe96ee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:10:44.835620 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-59e1367d-d14c-4c3f-ae96-3245eefe96ee service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:10:44.942827 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e1367d-d14c-4c3f-ae96-3245eefe96ee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:10:44.944249 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e1367d-d14c-4c3f-ae96-3245eefe96ee service neutron] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:10:44.962698 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 728/1461] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:10:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:10:46.916941 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c920a610-c2ae-475f-9cc2-116494e445b1 None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:10:46.917499 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c920a610-c2ae-475f-9cc2-116494e445b1 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:10:46.917499 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c920a610-c2ae-475f-9cc2-116494e445b1 None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:10:46.918838 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 734/1462] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 06:10:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:10:46.924260 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe8a906-532f-477c-86df-cb1e0b616f1c None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:10:46.924712 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe8a906-532f-477c-86df-cb1e0b616f1c None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:10:46.924829 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe8a906-532f-477c-86df-cb1e0b616f1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:10:47.040079 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-8fe8a906-532f-477c-86df-cb1e0b616f1c None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:10:47.040079 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-8fe8a906-532f-477c-86df-cb1e0b616f1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:10:47.040079 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:10:47.093513 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 729/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 06:10:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 06:10:47.375074 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c920a610-c2ae-475f-9cc2-116494e445b1 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:10:47.376667 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-c920a610-c2ae-475f-9cc2-116494e445b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:10:47.482435 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c920a610-c2ae-475f-9cc2-116494e445b1 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:10:47.521560 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:10:47.558109 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a53bb59-279c-4fba-bf4c-c06e44a019b3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bb6c79db75e4d169331c02133d1ec32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:10:47.558717 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a53bb59-279c-4fba-bf4c-c06e44a019b3 service neutron] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:10:47.558992 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a53bb59-279c-4fba-bf4c-c06e44a019b3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:10:47.559262 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a53bb59-279c-4fba-bf4c-c06e44a019b3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:10:47.560890 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-1a53bb59-279c-4fba-bf4c-c06e44a019b3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:10:47.678901 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a53bb59-279c-4fba-bf4c-c06e44a019b3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:10:47.680665 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a53bb59-279c-4fba-bf4c-c06e44a019b3 service neutron] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:10:47.702997 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 735/1464] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:10:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:10:49.607129 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a1be7b-9d84-400e-a9d9-1fab8229f8eb None None] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:10:49.607693 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a1be7b-9d84-400e-a9d9-1fab8229f8eb None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:10:49.607693 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a1be7b-9d84-400e-a9d9-1fab8229f8eb None None] PATH_INFO: `/` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:10:49.609027 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 730/1465] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 06:10:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:10:49.614608 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46eed1a2-4750-4379-810f-13da9b7995fa None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:10:49.615054 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46eed1a2-4750-4379-810f-13da9b7995fa None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:10:49.615171 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46eed1a2-4750-4379-810f-13da9b7995fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:10:49.685628 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-46eed1a2-4750-4379-810f-13da9b7995fa None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:10:49.686834 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-46eed1a2-4750-4379-810f-13da9b7995fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:10:49.706809 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:10:49.787888 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 736/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 06:10:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:10:50.079744 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a1be7b-9d84-400e-a9d9-1fab8229f8eb None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:10:50.081318 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-09a1be7b-9d84-400e-a9d9-1fab8229f8eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:10:50.185133 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a1be7b-9d84-400e-a9d9-1fab8229f8eb None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:10:50.253486 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:10:50.261022 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2d6652-ad4f-46e7-8293-4caa008a70c6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bb6c79db75e4d169331c02133d1ec32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:10:50.265027 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2d6652-ad4f-46e7-8293-4caa008a70c6 service neutron] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:10:50.265027 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2d6652-ad4f-46e7-8293-4caa008a70c6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:10:50.265027 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2d6652-ad4f-46e7-8293-4caa008a70c6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:10:50.265027 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-ca2d6652-ad4f-46e7-8293-4caa008a70c6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:10:50.374394 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2d6652-ad4f-46e7-8293-4caa008a70c6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:10:50.375829 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2d6652-ad4f-46e7-8293-4caa008a70c6 service neutron] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:10:50.393641 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 731/1467] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:10:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:10:52.288144 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952d5fe5-3dbd-4d65-9bc2-6610b064b577 None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:10:52.288758 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952d5fe5-3dbd-4d65-9bc2-6610b064b577 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:10:52.288758 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952d5fe5-3dbd-4d65-9bc2-6610b064b577 None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:10:52.290062 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 737/1468] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 06:10:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:10:52.295754 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c932b736-01bb-4eda-a7b0-edcb8c461f72 None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:10:52.296205 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c932b736-01bb-4eda-a7b0-edcb8c461f72 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:10:52.296318 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c932b736-01bb-4eda-a7b0-edcb8c461f72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:10:52.362758 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-c932b736-01bb-4eda-a7b0-edcb8c461f72 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:10:52.363961 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-c932b736-01bb-4eda-a7b0-edcb8c461f72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:10:52.412343 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:10:52.459885 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 732/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 06:10:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 06:10:52.742096 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952d5fe5-3dbd-4d65-9bc2-6610b064b577 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:10:52.744352 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-952d5fe5-3dbd-4d65-9bc2-6610b064b577 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:10:52.847761 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952d5fe5-3dbd-4d65-9bc2-6610b064b577 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:10:52.898892 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:10:52.925552 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f388b659-c6b8-4f19-b7a0-81f2bef89820 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bb6c79db75e4d169331c02133d1ec32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:10:52.926157 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f388b659-c6b8-4f19-b7a0-81f2bef89820 service neutron] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:10:52.926433 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f388b659-c6b8-4f19-b7a0-81f2bef89820 service neutron] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:10:52.926702 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f388b659-c6b8-4f19-b7a0-81f2bef89820 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:10:52.928310 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-f388b659-c6b8-4f19-b7a0-81f2bef89820 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:10:53.034075 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f388b659-c6b8-4f19-b7a0-81f2bef89820 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:10:53.035407 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f388b659-c6b8-4f19-b7a0-81f2bef89820 service neutron] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:10:53.053687 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 738/1470] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:10:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:11:02.458322 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=22 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:11:03.051440 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=66 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:11:12.758822 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce640f76-ed22-443a-8781-2a3ff6bd3665 None None] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:11:12.759334 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce640f76-ed22-443a-8781-2a3ff6bd3665 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:11:12.759334 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce640f76-ed22-443a-8781-2a3ff6bd3665 None None] PATH_INFO: `/` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:11:12.760729 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 733/1471] 10.222.0.62 () {54 vars in 927 bytes} [Wed Aug 24 06:11:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:11:12.765929 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3dc3fe-3ac1-4cc7-bfa3-96e04bc7a7b3 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:11:12.766350 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3dc3fe-3ac1-4cc7-bfa3-96e04bc7a7b3 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:11:12.766482 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3dc3fe-3ac1-4cc7-bfa3-96e04bc7a7b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:11:12.831723 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-fd3dc3fe-3ac1-4cc7-bfa3-96e04bc7a7b3 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:11:12.832959 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-fd3dc3fe-3ac1-4cc7-bfa3-96e04bc7a7b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:11:12.933232 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 739/1472] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Aug 24 06:11:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:11:13.227945 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2223a60-3d9a-4a8a-9902-45ff83e1115b None None] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:11:13.228231 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2223a60-3d9a-4a8a-9902-45ff83e1115b None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:11:13.228536 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2223a60-3d9a-4a8a-9902-45ff83e1115b None None] PATH_INFO: `/` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:11:13.230170 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 734/1473] 10.222.0.62 () {54 vars in 967 bytes} [Wed Aug 24 06:11:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:11:13.235118 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d782a9-ab46-4e2a-ad34-5b75f2c84a4e None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:11:13.235401 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d782a9-ab46-4e2a-ad34-5b75f2c84a4e None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:11:13.235671 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d782a9-ab46-4e2a-ad34-5b75f2c84a4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:11:13.310820 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-a0d782a9-ab46-4e2a-ad34-5b75f2c84a4e None None] MFA Rules not processed for user `c7d52162cbdc4218b577ecc1e69a88c9`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:11:13.312300 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-a0d782a9-ab46-4e2a-ad34-5b75f2c84a4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:11:13.333227 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:11:13.387353 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 740/1474] 10.222.0.62 () {58 vars in 1064 bytes} [Wed Aug 24 06:11:13 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:11:13.395099 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2223a60-3d9a-4a8a-9902-45ff83e1115b None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:11:13.396688 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-d2223a60-3d9a-4a8a-9902-45ff83e1115b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:11:13.498778 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2223a60-3d9a-4a8a-9902-45ff83e1115b None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:11:13.572053 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d940db1-7294-493c-b180-69d4660fc112 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7d52162cbdc4218b577ecc1e69a88c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:11:13.572689 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d940db1-7294-493c-b180-69d4660fc112 service glance] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:11:13.572966 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d940db1-7294-493c-b180-69d4660fc112 service glance] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:11:13.573234 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d940db1-7294-493c-b180-69d4660fc112 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:11:13.574809 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-0d940db1-7294-493c-b180-69d4660fc112 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:11:13.629184 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:11:13.687099 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d940db1-7294-493c-b180-69d4660fc112 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:11:13.688577 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d940db1-7294-493c-b180-69d4660fc112 service glance] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:11:13.708442 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 735/1475] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:11:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:11:15.729308 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10df848c-608c-422d-a257-d6f1fdcc6b37 None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:11:15.729850 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10df848c-608c-422d-a257-d6f1fdcc6b37 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:11:15.729850 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10df848c-608c-422d-a257-d6f1fdcc6b37 None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:11:15.731215 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 741/1476] 10.222.0.62 () {54 vars in 927 bytes} [Wed Aug 24 06:11:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:11:15.737034 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ee851a-d2aa-424e-a89f-47aa1ac0dfcb None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:11:15.737493 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ee851a-d2aa-424e-a89f-47aa1ac0dfcb None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:11:15.737593 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ee851a-d2aa-424e-a89f-47aa1ac0dfcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:11:15.809440 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-f1ee851a-d2aa-424e-a89f-47aa1ac0dfcb None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:11:15.810656 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-f1ee851a-d2aa-424e-a89f-47aa1ac0dfcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:11:15.907468 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 736/1477] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Aug 24 06:11:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:11:16.282560 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10df848c-608c-422d-a257-d6f1fdcc6b37 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:11:16.284132 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-10df848c-608c-422d-a257-d6f1fdcc6b37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:11:16.412371 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10df848c-608c-422d-a257-d6f1fdcc6b37 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:11:16.464818 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:11:16.510065 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f065056-b78c-4191-be8a-eebe51db5fa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:11:16.510662 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f065056-b78c-4191-be8a-eebe51db5fa2 admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:11:16.510935 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f065056-b78c-4191-be8a-eebe51db5fa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:11:16.511199 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f065056-b78c-4191-be8a-eebe51db5fa2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:11:16.511821 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f065056-b78c-4191-be8a-eebe51db5fa2 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:11:16.513578 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: WARNING py.warnings [None req-8f065056-b78c-4191-be8a-eebe51db5fa2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 06:11:16.513578 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: warnings.warn(msg) Aug 24 06:11:16.513578 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: Aug 24 06:11:16.514727 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f065056-b78c-4191-be8a-eebe51db5fa2 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:11:16.534133 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 742/1478] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Aug 24 06:11:16 2022] GET /identity/v3/projects => generated 2553 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:11:18.514890 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96751d01-297b-464e-975c-7f92cb8a8cdd None None] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:11:18.515559 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96751d01-297b-464e-975c-7f92cb8a8cdd None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:11:18.515559 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96751d01-297b-464e-975c-7f92cb8a8cdd None None] PATH_INFO: `/` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:11:18.516795 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 737/1479] 10.222.0.62 () {54 vars in 927 bytes} [Wed Aug 24 06:11:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:11:18.522154 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccdc91c-af5b-4579-8441-ee0234efc3b1 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:11:18.522586 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccdc91c-af5b-4579-8441-ee0234efc3b1 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:11:18.522713 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccdc91c-af5b-4579-8441-ee0234efc3b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:11:18.588707 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-9ccdc91c-af5b-4579-8441-ee0234efc3b1 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:11:18.589810 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-9ccdc91c-af5b-4579-8441-ee0234efc3b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:11:18.683134 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 743/1480] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Aug 24 06:11:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:11:18.973445 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96751d01-297b-464e-975c-7f92cb8a8cdd None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:11:18.974993 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-96751d01-297b-464e-975c-7f92cb8a8cdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:11:18.985441 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:11:19.126067 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96751d01-297b-464e-975c-7f92cb8a8cdd None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:11:19.209882 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:11:19.246658 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b637e1-4f26-4095-8543-dd3855b858ea service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb6d3743340241f98cacb9e0d063298c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:11:19.247354 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b637e1-4f26-4095-8543-dd3855b858ea service nova] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:11:19.247658 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b637e1-4f26-4095-8543-dd3855b858ea service nova] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:11:19.247951 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b637e1-4f26-4095-8543-dd3855b858ea service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:11:19.249771 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-79b637e1-4f26-4095-8543-dd3855b858ea service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:11:19.341876 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b637e1-4f26-4095-8543-dd3855b858ea service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:11:19.343506 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b637e1-4f26-4095-8543-dd3855b858ea service nova] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:11:19.361536 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 738/1481] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:11:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:11:21.448019 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e70af0-7dcc-4d63-bb87-8104a752a865 None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:11:21.448639 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e70af0-7dcc-4d63-bb87-8104a752a865 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:11:21.448639 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e70af0-7dcc-4d63-bb87-8104a752a865 None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:11:21.449925 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 744/1482] 10.222.0.62 () {54 vars in 927 bytes} [Wed Aug 24 06:11:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:11:21.455344 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba3e0f8-6565-4ae2-873e-35dccbed1003 None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:11:21.455800 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba3e0f8-6565-4ae2-873e-35dccbed1003 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:11:21.455905 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba3e0f8-6565-4ae2-873e-35dccbed1003 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:11:21.521218 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-dba3e0f8-6565-4ae2-873e-35dccbed1003 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:11:21.522404 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-dba3e0f8-6565-4ae2-873e-35dccbed1003 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:11:21.568393 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:11:21.619050 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 739/1483] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Aug 24 06:11:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:11:21.909294 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e70af0-7dcc-4d63-bb87-8104a752a865 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:11:21.910823 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-d9e70af0-7dcc-4d63-bb87-8104a752a865 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:11:21.934671 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:11:22.022864 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e70af0-7dcc-4d63-bb87-8104a752a865 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:11:22.100230 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ad07b5-0eba-49c9-b59f-a2d3de0340e9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bb6c79db75e4d169331c02133d1ec32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:11:22.100857 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ad07b5-0eba-49c9-b59f-a2d3de0340e9 service neutron] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:11:22.101140 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ad07b5-0eba-49c9-b59f-a2d3de0340e9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:11:22.101409 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ad07b5-0eba-49c9-b59f-a2d3de0340e9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:11:22.102993 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-22ad07b5-0eba-49c9-b59f-a2d3de0340e9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:11:22.176106 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:11:22.211099 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ad07b5-0eba-49c9-b59f-a2d3de0340e9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:11:22.212644 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ad07b5-0eba-49c9-b59f-a2d3de0340e9 service neutron] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:11:22.230170 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 745/1484] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:11:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:11:24.201565 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14b5ac7-34b9-4aa9-b736-bebc3cfb9881 None None] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:11:24.202144 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14b5ac7-34b9-4aa9-b736-bebc3cfb9881 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:11:24.202144 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14b5ac7-34b9-4aa9-b736-bebc3cfb9881 None None] PATH_INFO: `/` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:11:24.203443 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 740/1485] 10.222.0.62 () {54 vars in 927 bytes} [Wed Aug 24 06:11:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:11:24.208807 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4ca61c-340e-460c-921a-a9bdb830fe99 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:11:24.209226 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4ca61c-340e-460c-921a-a9bdb830fe99 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:11:24.209350 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4ca61c-340e-460c-921a-a9bdb830fe99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:11:24.273406 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-5e4ca61c-340e-460c-921a-a9bdb830fe99 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:11:24.274604 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-5e4ca61c-340e-460c-921a-a9bdb830fe99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:11:24.368017 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 746/1486] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Aug 24 06:11:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:11:24.654893 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14b5ac7-34b9-4aa9-b736-bebc3cfb9881 None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:11:24.656459 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-a14b5ac7-34b9-4aa9-b736-bebc3cfb9881 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:11:24.894798 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14b5ac7-34b9-4aa9-b736-bebc3cfb9881 None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:11:24.894798 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:11:24.894798 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6270eb65-5aab-4a8b-b798-b74a2fb412cd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bb6c79db75e4d169331c02133d1ec32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:11:24.894798 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6270eb65-5aab-4a8b-b798-b74a2fb412cd service neutron] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:11:24.895303 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6270eb65-5aab-4a8b-b798-b74a2fb412cd service neutron] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:11:24.895303 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6270eb65-5aab-4a8b-b798-b74a2fb412cd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:11:24.895303 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-6270eb65-5aab-4a8b-b798-b74a2fb412cd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:11:24.959123 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6270eb65-5aab-4a8b-b798-b74a2fb412cd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:11:24.960582 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6270eb65-5aab-4a8b-b798-b74a2fb412cd service neutron] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:11:24.979125 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 741/1487] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:11:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:11:27.035903 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd0bbe8-eff4-4b76-8b7d-82dd4ad92ba8 None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:11:27.036444 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd0bbe8-eff4-4b76-8b7d-82dd4ad92ba8 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:11:27.036444 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd0bbe8-eff4-4b76-8b7d-82dd4ad92ba8 None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:11:27.037759 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 747/1488] 10.222.0.62 () {54 vars in 927 bytes} [Wed Aug 24 06:11:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:11:27.043623 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8c6f52-8710-44b5-b8a4-3f424f25a141 None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:11:27.045318 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8c6f52-8710-44b5-b8a4-3f424f25a141 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:11:27.045318 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8c6f52-8710-44b5-b8a4-3f424f25a141 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:11:27.112165 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-dd8c6f52-8710-44b5-b8a4-3f424f25a141 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:11:27.113401 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-dd8c6f52-8710-44b5-b8a4-3f424f25a141 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:11:27.171502 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:11:27.218715 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 742/1489] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Aug 24 06:11:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:11:27.512243 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd0bbe8-eff4-4b76-8b7d-82dd4ad92ba8 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:11:27.513816 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-ecd0bbe8-eff4-4b76-8b7d-82dd4ad92ba8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:11:27.548450 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:11:27.618031 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd0bbe8-eff4-4b76-8b7d-82dd4ad92ba8 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:11:27.690968 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209fd5b8-43a4-445f-9d1f-6d71deb87934 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bb6c79db75e4d169331c02133d1ec32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:11:27.691582 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209fd5b8-43a4-445f-9d1f-6d71deb87934 service neutron] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:11:27.691861 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209fd5b8-43a4-445f-9d1f-6d71deb87934 service neutron] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:11:27.692133 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209fd5b8-43a4-445f-9d1f-6d71deb87934 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:11:27.693796 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-209fd5b8-43a4-445f-9d1f-6d71deb87934 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:11:27.774744 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:11:27.803604 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209fd5b8-43a4-445f-9d1f-6d71deb87934 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:11:27.805895 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209fd5b8-43a4-445f-9d1f-6d71deb87934 service neutron] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:11:27.824013 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 748/1490] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:11:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:11:30.107035 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce5a2de-cb03-41d0-83f1-4383365a8982 None None] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:11:30.107588 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce5a2de-cb03-41d0-83f1-4383365a8982 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:11:30.107588 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce5a2de-cb03-41d0-83f1-4383365a8982 None None] PATH_INFO: `/` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:11:30.109021 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 743/1491] 10.222.0.62 () {54 vars in 927 bytes} [Wed Aug 24 06:11:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:11:30.114156 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0ec01a-3fcf-4b7d-bb26-b7a37fcf5bde None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:11:30.114606 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0ec01a-3fcf-4b7d-bb26-b7a37fcf5bde None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:11:30.114714 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0ec01a-3fcf-4b7d-bb26-b7a37fcf5bde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:11:30.180230 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-1a0ec01a-3fcf-4b7d-bb26-b7a37fcf5bde None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:11:30.181521 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-1a0ec01a-3fcf-4b7d-bb26-b7a37fcf5bde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:11:30.276629 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 749/1492] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Aug 24 06:11:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:11:30.563125 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce5a2de-cb03-41d0-83f1-4383365a8982 None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:11:30.564698 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-9ce5a2de-cb03-41d0-83f1-4383365a8982 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:11:30.668340 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce5a2de-cb03-41d0-83f1-4383365a8982 None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:11:30.740952 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc36c98-3760-4318-8639-d5a16bf8d79b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bb6c79db75e4d169331c02133d1ec32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:11:30.741549 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc36c98-3760-4318-8639-d5a16bf8d79b service neutron] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:11:30.741825 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc36c98-3760-4318-8639-d5a16bf8d79b service neutron] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:11:30.742095 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc36c98-3760-4318-8639-d5a16bf8d79b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:11:30.743685 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-5fc36c98-3760-4318-8639-d5a16bf8d79b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:11:30.752524 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:11:30.851835 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc36c98-3760-4318-8639-d5a16bf8d79b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:11:30.853234 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc36c98-3760-4318-8639-d5a16bf8d79b service neutron] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:11:30.870748 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 744/1493] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:11:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:11:40.274855 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=82 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:11:40.868472 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=50 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:12:51.823691 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bb1efc-5b46-4ebf-8577-a76acf8b4d03 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:12:51.824297 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bb1efc-5b46-4ebf-8577-a76acf8b4d03 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:12:51.824297 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bb1efc-5b46-4ebf-8577-a76acf8b4d03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:12:51.898873 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-41bb1efc-5b46-4ebf-8577-a76acf8b4d03 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:12:51.900647 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-41bb1efc-5b46-4ebf-8577-a76acf8b4d03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:12:51.998070 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 750/1494] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:12:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:12:52.006328 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6902193-d716-4a3f-8c23-f628a54a5ccd None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:12:52.006850 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6902193-d716-4a3f-8c23-f628a54a5ccd None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:12:52.006850 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6902193-d716-4a3f-8c23-f628a54a5ccd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:12:52.083096 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-b6902193-d716-4a3f-8c23-f628a54a5ccd None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:12:52.084291 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-b6902193-d716-4a3f-8c23-f628a54a5ccd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:12:52.181188 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 745/1495] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:12:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:12:52.185851 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bb1efc-5b46-4ebf-8577-a76acf8b4d03 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:12:52.187044 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-41bb1efc-5b46-4ebf-8577-a76acf8b4d03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:12:52.198729 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bb1efc-5b46-4ebf-8577-a76acf8b4d03 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:12:52.258200 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:12:52.291975 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab32d6d-6ce1-4ee1-8186-502e30676b2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:12:52.292604 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab32d6d-6ce1-4ee1-8186-502e30676b2c admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:12:52.292878 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab32d6d-6ce1-4ee1-8186-502e30676b2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:12:52.293142 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab32d6d-6ce1-4ee1-8186-502e30676b2c admin admin] PATH_INFO: `/v3/domains` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:12:52.293770 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab32d6d-6ce1-4ee1-8186-502e30676b2c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:12:52.294069 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab32d6d-6ce1-4ee1-8186-502e30676b2c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:12:52.295073 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: WARNING py.warnings [None req-0ab32d6d-6ce1-4ee1-8186-502e30676b2c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 24 06:12:52.295073 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: warnings.warn(msg) Aug 24 06:12:52.295073 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: Aug 24 06:12:52.295962 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab32d6d-6ce1-4ee1-8186-502e30676b2c admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:12:52.308160 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 751/1496] 10.222.0.62 () {58 vars in 1153 bytes} [Wed Aug 24 06:12:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 06:12:52.316737 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6902193-d716-4a3f-8c23-f628a54a5ccd None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:12:52.326647 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6902193-d716-4a3f-8c23-f628a54a5ccd None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:12:52.387894 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:12:52.420405 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c65a71-5fe9-471b-a629-d0c33e83bec1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:12:52.421056 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c65a71-5fe9-471b-a629-d0c33e83bec1 admin admin] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:12:52.421331 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c65a71-5fe9-471b-a629-d0c33e83bec1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:12:52.421599 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c65a71-5fe9-471b-a629-d0c33e83bec1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:12:52.422220 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c65a71-5fe9-471b-a629-d0c33e83bec1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:12:52.423933 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: WARNING py.warnings [None req-25c65a71-5fe9-471b-a629-d0c33e83bec1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 06:12:52.423933 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: warnings.warn(msg) Aug 24 06:12:52.423933 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: Aug 24 06:12:52.425397 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c65a71-5fe9-471b-a629-d0c33e83bec1 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:12:52.554812 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 746/1497] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Aug 24 06:12:52 2022] POST /identity/v3/projects => generated 377 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:12:52.559688 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab32d6d-6ce1-4ee1-8186-502e30676b2c admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:12:52.561279 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-0ab32d6d-6ce1-4ee1-8186-502e30676b2c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:12:52.595613 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab32d6d-6ce1-4ee1-8186-502e30676b2c admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:12:52.705977 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c1e556-9ef8-4509-a666-fc15cd9828c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:12:52.706587 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c1e556-9ef8-4509-a666-fc15cd9828c0 admin admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:12:52.706848 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c1e556-9ef8-4509-a666-fc15cd9828c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:12:52.707116 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c1e556-9ef8-4509-a666-fc15cd9828c0 admin admin] PATH_INFO: `/v3/users` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:12:52.708400 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c1e556-9ef8-4509-a666-fc15cd9828c0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:12:52.708819 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: WARNING py.warnings [None req-91c1e556-9ef8-4509-a666-fc15cd9828c0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 06:12:52.708819 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: warnings.warn(msg) Aug 24 06:12:52.708819 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: Aug 24 06:12:52.710029 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c1e556-9ef8-4509-a666-fc15cd9828c0 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:12:52.844781 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 752/1498] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Aug 24 06:12:52 2022] POST /identity/v3/users => generated 289 bytes in 285 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:12:52.849087 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c65a71-5fe9-471b-a629-d0c33e83bec1 admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:12:52.850686 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-25c65a71-5fe9-471b-a629-d0c33e83bec1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:12:52.914454 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c65a71-5fe9-471b-a629-d0c33e83bec1 admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:12:52.999356 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:12:53.010112 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4119565-770e-472b-a457-2fbb78a8e4fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:12:53.010767 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4119565-770e-472b-a457-2fbb78a8e4fc admin admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:12:53.010979 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4119565-770e-472b-a457-2fbb78a8e4fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:12:53.011246 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4119565-770e-472b-a457-2fbb78a8e4fc admin admin] PATH_INFO: `/v3/roles` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:12:53.011895 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4119565-770e-472b-a457-2fbb78a8e4fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:12:53.012907 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: WARNING py.warnings [None req-e4119565-770e-472b-a457-2fbb78a8e4fc admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 24 06:12:53.012907 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: warnings.warn(msg) Aug 24 06:12:53.012907 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: Aug 24 06:12:53.013792 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4119565-770e-472b-a457-2fbb78a8e4fc admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:12:53.026707 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 747/1499] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Aug 24 06:12:52 2022] GET /identity/v3/roles => generated 1420 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:12:53.030861 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c1e556-9ef8-4509-a666-fc15cd9828c0 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:12:53.032362 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-91c1e556-9ef8-4509-a666-fc15cd9828c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:12:53.044750 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c1e556-9ef8-4509-a666-fc15cd9828c0 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:12:53.070253 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:12:53.140276 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aaab4a5-4c8f-481b-8ce4-b1035754626d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:12:53.140855 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aaab4a5-4c8f-481b-8ce4-b1035754626d admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:12:53.141134 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aaab4a5-4c8f-481b-8ce4-b1035754626d admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:12:53.141397 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aaab4a5-4c8f-481b-8ce4-b1035754626d admin admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:12:53.142023 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aaab4a5-4c8f-481b-8ce4-b1035754626d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:12:53.143023 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: WARNING py.warnings [None req-3aaab4a5-4c8f-481b-8ce4-b1035754626d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 24 06:12:53.143023 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: warnings.warn(msg) Aug 24 06:12:53.143023 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: Aug 24 06:12:53.143908 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aaab4a5-4c8f-481b-8ce4-b1035754626d admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:12:53.156660 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 753/1500] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Aug 24 06:12:53 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 06:12:53.161058 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4119565-770e-472b-a457-2fbb78a8e4fc admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:12:53.162570 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-e4119565-770e-472b-a457-2fbb78a8e4fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:12:53.195646 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4119565-770e-472b-a457-2fbb78a8e4fc admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:12:53.292226 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0564eb62-ee74-4f72-8c4e-2217b5ca2a60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:12:53.292889 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0564eb62-ee74-4f72-8c4e-2217b5ca2a60 admin admin] REQUEST_METHOD: `PUT` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:12:53.293173 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0564eb62-ee74-4f72-8c4e-2217b5ca2a60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:12:53.293445 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0564eb62-ee74-4f72-8c4e-2217b5ca2a60 admin admin] PATH_INFO: `/v3/projects/32ab744e5eb641f0bc368c371b5a30f0/users/efc3b21a205042b09c46878388434f03/roles/580aa7dbb38d4cff8f9d1bb8983f99ef` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:12:53.315580 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0564eb62-ee74-4f72-8c4e-2217b5ca2a60 admin admin] RBAC: Authorizing `identity:create_grant(project_id=32ab744e5eb641f0bc368c371b5a30f0, user_id=efc3b21a205042b09c46878388434f03, role_id=580aa7dbb38d4cff8f9d1bb8983f99ef)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:12:53.316635 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: WARNING py.warnings [None req-0564eb62-ee74-4f72-8c4e-2217b5ca2a60 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 06:12:53.316635 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: warnings.warn(msg) Aug 24 06:12:53.316635 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: Aug 24 06:12:53.318578 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0564eb62-ee74-4f72-8c4e-2217b5ca2a60 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:12:53.438558 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 748/1501] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Aug 24 06:12:53 2022] PUT /identity/v3/projects/32ab744e5eb641f0bc368c371b5a30f0/users/efc3b21a205042b09c46878388434f03/roles/580aa7dbb38d4cff8f9d1bb8983f99ef => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:12:53.450960 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf65bb5-b9d4-4a8d-bfd5-0bd143177efe None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:12:53.451390 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf65bb5-b9d4-4a8d-bfd5-0bd143177efe None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:12:53.451757 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf65bb5-b9d4-4a8d-bfd5-0bd143177efe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:12:53.507996 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-8bf65bb5-b9d4-4a8d-bfd5-0bd143177efe None None] MFA Rules not processed for user `efc3b21a205042b09c46878388434f03`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:12:53.509246 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-8bf65bb5-b9d4-4a8d-bfd5-0bd143177efe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:12:53.586071 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 754/1502] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:12:53 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:12:53.599688 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e1970a-c452-4be1-991f-706ea0973c22 None None] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:12:53.599976 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e1970a-c452-4be1-991f-706ea0973c22 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:12:53.600283 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e1970a-c452-4be1-991f-706ea0973c22 None None] PATH_INFO: `/` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:12:53.601645 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 749/1503] 10.222.0.62 () {54 vars in 964 bytes} [Wed Aug 24 06:12:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:12:53.607410 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0f8598-beab-42d0-ba06-39d648a632d6 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:12:53.607737 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0f8598-beab-42d0-ba06-39d648a632d6 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:12:53.608050 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0f8598-beab-42d0-ba06-39d648a632d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:12:53.639905 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:12:53.733609 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-0a0f8598-beab-42d0-ba06-39d648a632d6 None None] MFA Rules not processed for user `8c4def55ac0e48b29836bc5c64bc7402`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:12:53.734807 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-0a0f8598-beab-42d0-ba06-39d648a632d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:12:53.810106 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 755/1504] 10.222.0.62 () {58 vars in 1061 bytes} [Wed Aug 24 06:12:53 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 06:12:53.817966 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e1970a-c452-4be1-991f-706ea0973c22 None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:12:53.819487 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-b3e1970a-c452-4be1-991f-706ea0973c22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:12:53.872701 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:12:53.927169 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e1970a-c452-4be1-991f-706ea0973c22 None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:12:54.002969 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b83ea4-d6b3-428e-991c-14b2b5974127 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c4def55ac0e48b29836bc5c64bc7402', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:12:54.003491 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b83ea4-d6b3-428e-991c-14b2b5974127 service cinder] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:12:54.003767 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b83ea4-d6b3-428e-991c-14b2b5974127 service cinder] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:12:54.004034 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b83ea4-d6b3-428e-991c-14b2b5974127 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:12:54.005665 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-b0b83ea4-d6b3-428e-991c-14b2b5974127 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:12:54.103330 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b83ea4-d6b3-428e-991c-14b2b5974127 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:12:54.104811 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b83ea4-d6b3-428e-991c-14b2b5974127 service cinder] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:12:54.108348 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:12:54.123493 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 750/1505] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:12:53 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:12:54.169845 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bd7f8a-423b-428f-85fd-0047bf9ad383 None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:12:54.170398 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bd7f8a-423b-428f-85fd-0047bf9ad383 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:12:54.170540 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bd7f8a-423b-428f-85fd-0047bf9ad383 None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:12:54.171867 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 756/1506] 10.222.0.62 () {52 vars in 830 bytes} [Wed Aug 24 06:12:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:12:54.175889 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b83ea4-d6b3-428e-991c-14b2b5974127 service cinder] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:12:54.177236 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-b0b83ea4-d6b3-428e-991c-14b2b5974127 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:12:54.275211 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b83ea4-d6b3-428e-991c-14b2b5974127 service cinder] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:12:54.368179 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a0d9ef-bf78-4c99-bd8c-32e4ed422e47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:12:54.368842 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a0d9ef-bf78-4c99-bd8c-32e4ed422e47 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:12:54.369119 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a0d9ef-bf78-4c99-bd8c-32e4ed422e47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:12:54.369391 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a0d9ef-bf78-4c99-bd8c-32e4ed422e47 admin admin] PATH_INFO: `/v3/users/efc3b21a205042b09c46878388434f03` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:12:54.381757 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a0d9ef-bf78-4c99-bd8c-32e4ed422e47 admin admin] RBAC: Authorizing `identity:delete_user(user_id=efc3b21a205042b09c46878388434f03)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:12:54.382779 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: WARNING py.warnings [None req-39a0d9ef-bf78-4c99-bd8c-32e4ed422e47 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 06:12:54.382779 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: warnings.warn(msg) Aug 24 06:12:54.382779 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: Aug 24 06:12:54.384007 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a0d9ef-bf78-4c99-bd8c-32e4ed422e47 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:12:54.393824 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:12:54.566557 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-39a0d9ef-bf78-4c99-bd8c-32e4ed422e47 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'efc3b21a205042b09c46878388434f03'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:12:54.581042 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-39a0d9ef-bf78-4c99-bd8c-32e4ed422e47 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'efc3b21a205042b09c46878388434f03'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:12:54.581620 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-39a0d9ef-bf78-4c99-bd8c-32e4ed422e47 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'efc3b21a205042b09c46878388434f03'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:12:54.582047 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-39a0d9ef-bf78-4c99-bd8c-32e4ed422e47 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'efc3b21a205042b09c46878388434f03'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:12:54.647587 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-39a0d9ef-bf78-4c99-bd8c-32e4ed422e47 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'efc3b21a205042b09c46878388434f03'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:12:54.677332 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 751/1507] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Aug 24 06:12:54 2022] DELETE /identity/v3/users/efc3b21a205042b09c46878388434f03 => generated 0 bytes in 502 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:12:54.682071 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f53bffe-13d3-497d-95b5-58c001021d0e None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:12:54.682355 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f53bffe-13d3-497d-95b5-58c001021d0e None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:12:54.682629 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f53bffe-13d3-497d-95b5-58c001021d0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:12:54.740642 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-0f53bffe-13d3-497d-95b5-58c001021d0e None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:12:54.741859 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-0f53bffe-13d3-497d-95b5-58c001021d0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:12:54.817905 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:12:54.831222 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 757/1508] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:12:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:12:54.841624 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a0d9ef-bf78-4c99-bd8c-32e4ed422e47 admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:12:54.842950 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-39a0d9ef-bf78-4c99-bd8c-32e4ed422e47 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:12:54.945326 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a0d9ef-bf78-4c99-bd8c-32e4ed422e47 admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:12:55.000463 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:12:55.018724 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99a2929-b462-47cb-bb03-8f3248c7502d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bb6c79db75e4d169331c02133d1ec32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:12:55.019335 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99a2929-b462-47cb-bb03-8f3248c7502d service neutron] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:12:55.019612 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99a2929-b462-47cb-bb03-8f3248c7502d service neutron] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:12:55.019881 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99a2929-b462-47cb-bb03-8f3248c7502d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:12:55.021502 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-a99a2929-b462-47cb-bb03-8f3248c7502d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:12:55.132006 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99a2929-b462-47cb-bb03-8f3248c7502d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:12:55.133395 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99a2929-b462-47cb-bb03-8f3248c7502d service neutron] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:12:55.152154 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 752/1509] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:12:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:12:55.626347 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f53bffe-13d3-497d-95b5-58c001021d0e None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:12:55.627906 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-0f53bffe-13d3-497d-95b5-58c001021d0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:12:55.642219 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f53bffe-13d3-497d-95b5-58c001021d0e None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:12:55.747005 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76380e0b-99eb-4030-9230-a83a2cb32970 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:12:55.747635 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76380e0b-99eb-4030-9230-a83a2cb32970 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:12:55.747913 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76380e0b-99eb-4030-9230-a83a2cb32970 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:12:55.748184 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76380e0b-99eb-4030-9230-a83a2cb32970 admin admin] PATH_INFO: `/v3/projects/32ab744e5eb641f0bc368c371b5a30f0` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:12:55.758029 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76380e0b-99eb-4030-9230-a83a2cb32970 admin admin] RBAC: Authorizing `identity:delete_project(project_id=32ab744e5eb641f0bc368c371b5a30f0)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:12:55.759097 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: WARNING py.warnings [None req-76380e0b-99eb-4030-9230-a83a2cb32970 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 06:12:55.759097 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: warnings.warn(msg) Aug 24 06:12:55.759097 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: Aug 24 06:12:55.760299 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76380e0b-99eb-4030-9230-a83a2cb32970 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:12:55.873146 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-76380e0b-99eb-4030-9230-a83a2cb32970 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '32ab744e5eb641f0bc368c371b5a30f0', 'request_id': 'req-76380e0b-99eb-4030-9230-a83a2cb32970'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:12:55.879096 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-76380e0b-99eb-4030-9230-a83a2cb32970 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '32ab744e5eb641f0bc368c371b5a30f0', 'request_id': 'req-76380e0b-99eb-4030-9230-a83a2cb32970'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:12:55.891895 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-76380e0b-99eb-4030-9230-a83a2cb32970 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '32ab744e5eb641f0bc368c371b5a30f0', 'request_id': 'req-76380e0b-99eb-4030-9230-a83a2cb32970'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:12:55.998119 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-76380e0b-99eb-4030-9230-a83a2cb32970 admin admin] The token cache is being invalidate because project 32ab744e5eb641f0bc368c371b5a30f0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81852) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:12:55.998353 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-76380e0b-99eb-4030-9230-a83a2cb32970 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:12:55.998840 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-76380e0b-99eb-4030-9230-a83a2cb32970 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:12:56.000148 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 758/1510] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Aug 24 06:12:55 2022] DELETE /identity/v3/projects/32ab744e5eb641f0bc368c371b5a30f0 => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:13:05.149638 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=62 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:13:05.995883 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:14:27.575657 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20bf799-54fe-46d2-a8a7-c5f6e3c9b2c4 None None] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:14:27.576279 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20bf799-54fe-46d2-a8a7-c5f6e3c9b2c4 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:14:27.576279 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20bf799-54fe-46d2-a8a7-c5f6e3c9b2c4 None None] PATH_INFO: `/` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:14:27.577595 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 753/1511] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 06:14:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:14:27.582505 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca13f509-acc3-4fd5-abc7-0cbfad8d80ca None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:14:27.583086 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca13f509-acc3-4fd5-abc7-0cbfad8d80ca None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:14:27.583086 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca13f509-acc3-4fd5-abc7-0cbfad8d80ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:14:27.663179 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-ca13f509-acc3-4fd5-abc7-0cbfad8d80ca None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:14:27.664331 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-ca13f509-acc3-4fd5-abc7-0cbfad8d80ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:14:27.766996 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 759/1512] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 06:14:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:14:27.920157 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20bf799-54fe-46d2-a8a7-c5f6e3c9b2c4 None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:14:27.922442 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-b20bf799-54fe-46d2-a8a7-c5f6e3c9b2c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:14:28.040867 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20bf799-54fe-46d2-a8a7-c5f6e3c9b2c4 None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:14:28.046322 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abde0f7e-553b-4b12-ac34-3cc42eb81319 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb6d3743340241f98cacb9e0d063298c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:14:28.046930 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abde0f7e-553b-4b12-ac34-3cc42eb81319 service nova] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:14:28.047204 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abde0f7e-553b-4b12-ac34-3cc42eb81319 service nova] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:14:28.047474 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abde0f7e-553b-4b12-ac34-3cc42eb81319 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:14:28.055536 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abde0f7e-553b-4b12-ac34-3cc42eb81319 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:14:28.057155 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abde0f7e-553b-4b12-ac34-3cc42eb81319 service nova] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:14:28.066413 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 754/1513] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:14:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:14:28.894521 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7673d40b-7962-452d-a52f-ad1a0a6882a1 None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:14:28.895085 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7673d40b-7962-452d-a52f-ad1a0a6882a1 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:14:28.895085 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7673d40b-7962-452d-a52f-ad1a0a6882a1 None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:14:28.896407 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 760/1514] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 06:14:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:14:28.901149 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689005e5-2cd7-4fdc-b843-e8e1b859b135 None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:14:28.901560 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689005e5-2cd7-4fdc-b843-e8e1b859b135 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:14:28.901696 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689005e5-2cd7-4fdc-b843-e8e1b859b135 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:14:28.921792 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-689005e5-2cd7-4fdc-b843-e8e1b859b135 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:14:28.923048 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-689005e5-2cd7-4fdc-b843-e8e1b859b135 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:14:28.932090 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 755/1515] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 06:14:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:14:29.081087 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7673d40b-7962-452d-a52f-ad1a0a6882a1 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:14:29.094942 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7673d40b-7962-452d-a52f-ad1a0a6882a1 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:14:29.100793 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8dbef0-2512-459d-a1ba-ffd74dcb0961 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb6d3743340241f98cacb9e0d063298c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:14:29.101380 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8dbef0-2512-459d-a1ba-ffd74dcb0961 service nova] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:14:29.101656 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8dbef0-2512-459d-a1ba-ffd74dcb0961 service nova] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:14:29.101926 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8dbef0-2512-459d-a1ba-ffd74dcb0961 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:14:29.109876 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8dbef0-2512-459d-a1ba-ffd74dcb0961 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:14:29.112020 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8dbef0-2512-459d-a1ba-ffd74dcb0961 service nova] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:14:29.121113 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 761/1516] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:14:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:14:29.770397 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398358fe-d6e4-4c14-a74f-8f2f2fd10f99 None None] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:14:29.770674 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398358fe-d6e4-4c14-a74f-8f2f2fd10f99 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:14:29.770947 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398358fe-d6e4-4c14-a74f-8f2f2fd10f99 None None] PATH_INFO: `/v3` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:14:29.772251 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 756/1517] 10.222.0.20 () {54 vars in 925 bytes} [Wed Aug 24 06:14:29 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:14:29.778433 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fbbe0a-9039-423b-88d7-d14cda720e33 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:14:29.778587 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fbbe0a-9039-423b-88d7-d14cda720e33 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:14:29.778936 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fbbe0a-9039-423b-88d7-d14cda720e33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:14:29.838725 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-65fbbe0a-9039-423b-88d7-d14cda720e33 None None] MFA Rules not processed for user `abe9d8c9291845dfa4ef55034e5c9e32`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:14:29.840626 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-65fbbe0a-9039-423b-88d7-d14cda720e33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:14:29.860072 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:14:29.914849 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 762/1518] 10.222.0.20 () {58 vars in 1015 bytes} [Wed Aug 24 06:14:29 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:14:34.013969 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4344f48-1b07-4ecb-bafb-3118bc49b51f None None] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:14:34.014606 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4344f48-1b07-4ecb-bafb-3118bc49b51f None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:14:34.014606 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4344f48-1b07-4ecb-bafb-3118bc49b51f None None] PATH_INFO: `/` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:14:34.015929 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 757/1519] 10.222.0.62 () {54 vars in 961 bytes} [Wed Aug 24 06:14:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:14:34.020954 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c7964b-8217-46be-9749-3f6e6234379c None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:14:34.021436 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c7964b-8217-46be-9749-3f6e6234379c None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:14:34.021545 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c7964b-8217-46be-9749-3f6e6234379c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:14:34.040633 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-67c7964b-8217-46be-9749-3f6e6234379c None None] MFA Rules not processed for user `abe9d8c9291845dfa4ef55034e5c9e32`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:14:34.041852 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-67c7964b-8217-46be-9749-3f6e6234379c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:14:34.051033 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 763/1520] 10.222.0.62 () {58 vars in 1058 bytes} [Wed Aug 24 06:14:34 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:14:34.058646 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4344f48-1b07-4ecb-bafb-3118bc49b51f None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:14:34.072438 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4344f48-1b07-4ecb-bafb-3118bc49b51f None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:14:34.077344 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59120cd4-8c78-4408-9677-1f11f7e31508 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abe9d8c9291845dfa4ef55034e5c9e32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:14:34.077928 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59120cd4-8c78-4408-9677-1f11f7e31508 service placement] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:14:34.078205 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59120cd4-8c78-4408-9677-1f11f7e31508 service placement] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:14:34.078821 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59120cd4-8c78-4408-9677-1f11f7e31508 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:14:34.086797 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59120cd4-8c78-4408-9677-1f11f7e31508 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:14:34.088355 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59120cd4-8c78-4408-9677-1f11f7e31508 service placement] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:14:34.097705 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 758/1521] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:14:34 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:14:40.342077 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aecd409-3821-4973-884b-1222c49bb574 None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:14:40.342599 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aecd409-3821-4973-884b-1222c49bb574 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:14:40.342599 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aecd409-3821-4973-884b-1222c49bb574 None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:14:40.343970 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 764/1522] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 06:14:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:14:40.348720 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5016dd-ea14-4f48-b773-0012b055d10d None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:14:40.349158 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5016dd-ea14-4f48-b773-0012b055d10d None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:14:40.349268 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5016dd-ea14-4f48-b773-0012b055d10d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:14:40.368598 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-be5016dd-ea14-4f48-b773-0012b055d10d None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:14:40.369789 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-be5016dd-ea14-4f48-b773-0012b055d10d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:14:40.377535 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 759/1523] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 06:14:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:14:40.533055 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aecd409-3821-4973-884b-1222c49bb574 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:14:40.547510 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aecd409-3821-4973-884b-1222c49bb574 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:14:40.553993 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c5d636-a1f2-435f-b98a-0fb6f46bac2e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb6d3743340241f98cacb9e0d063298c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:14:40.553993 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c5d636-a1f2-435f-b98a-0fb6f46bac2e service nova] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:14:40.554341 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c5d636-a1f2-435f-b98a-0fb6f46bac2e service nova] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:14:40.554612 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c5d636-a1f2-435f-b98a-0fb6f46bac2e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:14:40.562500 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c5d636-a1f2-435f-b98a-0fb6f46bac2e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:14:40.564064 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c5d636-a1f2-435f-b98a-0fb6f46bac2e service nova] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:14:40.573957 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 765/1524] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:14:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:14:41.386524 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4ae941-1134-46a4-ac24-6f28750c9561 None None] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:14:41.387112 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4ae941-1134-46a4-ac24-6f28750c9561 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:14:41.387112 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4ae941-1134-46a4-ac24-6f28750c9561 None None] PATH_INFO: `/` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:14:41.388411 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 760/1525] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 06:14:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:14:41.393302 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583a66a2-2c75-492b-bd89-882fe05f7bf6 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:14:41.393732 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583a66a2-2c75-492b-bd89-882fe05f7bf6 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:14:41.393926 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583a66a2-2c75-492b-bd89-882fe05f7bf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:14:41.413187 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-583a66a2-2c75-492b-bd89-882fe05f7bf6 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:14:41.414372 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-583a66a2-2c75-492b-bd89-882fe05f7bf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:14:41.422044 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 766/1526] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 06:14:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 06:14:41.577400 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4ae941-1134-46a4-ac24-6f28750c9561 None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:14:41.591501 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4ae941-1134-46a4-ac24-6f28750c9561 None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:14:41.596957 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f33f1b-7b64-4304-8f49-89c182902a8e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb6d3743340241f98cacb9e0d063298c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:14:41.597548 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f33f1b-7b64-4304-8f49-89c182902a8e service nova] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:14:41.597827 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f33f1b-7b64-4304-8f49-89c182902a8e service nova] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:14:41.598100 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f33f1b-7b64-4304-8f49-89c182902a8e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:14:41.605918 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f33f1b-7b64-4304-8f49-89c182902a8e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:14:41.607498 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f33f1b-7b64-4304-8f49-89c182902a8e service nova] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:14:41.617139 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 761/1527] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:14:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:14:45.586157 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f5a33f-44c3-4a11-b53b-32c1f7d398e3 None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:14:45.586735 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f5a33f-44c3-4a11-b53b-32c1f7d398e3 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:14:45.586735 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f5a33f-44c3-4a11-b53b-32c1f7d398e3 None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:14:45.588268 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 767/1528] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 06:14:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:14:45.593778 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f004cf00-acce-4296-9d81-e9d8c5ec7730 None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:14:45.594247 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f004cf00-acce-4296-9d81-e9d8c5ec7730 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:14:45.594352 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f004cf00-acce-4296-9d81-e9d8c5ec7730 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:14:45.615458 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-f004cf00-acce-4296-9d81-e9d8c5ec7730 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:14:45.616674 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-f004cf00-acce-4296-9d81-e9d8c5ec7730 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:14:45.624368 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 762/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 06:14:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:14:45.910731 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f5a33f-44c3-4a11-b53b-32c1f7d398e3 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:14:45.913756 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-f0f5a33f-44c3-4a11-b53b-32c1f7d398e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:14:45.991744 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f5a33f-44c3-4a11-b53b-32c1f7d398e3 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:14:45.995305 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaacaf9e-00d5-4918-82b1-c1517bde7983 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bb6c79db75e4d169331c02133d1ec32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:14:45.995900 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaacaf9e-00d5-4918-82b1-c1517bde7983 service neutron] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:14:45.996212 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaacaf9e-00d5-4918-82b1-c1517bde7983 service neutron] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:14:45.996476 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaacaf9e-00d5-4918-82b1-c1517bde7983 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:14:46.005375 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaacaf9e-00d5-4918-82b1-c1517bde7983 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:14:46.006800 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaacaf9e-00d5-4918-82b1-c1517bde7983 service neutron] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:14:46.016029 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 768/1530] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:14:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:14:47.895743 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9184c9-7111-484f-b03a-1f166d77a347 None None] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:14:47.896306 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9184c9-7111-484f-b03a-1f166d77a347 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:14:47.896306 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9184c9-7111-484f-b03a-1f166d77a347 None None] PATH_INFO: `/` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:14:47.897635 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 763/1531] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 06:14:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:14:47.902738 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d882043-0c66-4670-bc5b-9a0e71fb1490 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:14:47.903155 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d882043-0c66-4670-bc5b-9a0e71fb1490 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:14:47.903289 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d882043-0c66-4670-bc5b-9a0e71fb1490 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:14:47.922745 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-8d882043-0c66-4670-bc5b-9a0e71fb1490 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:14:47.923962 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-8d882043-0c66-4670-bc5b-9a0e71fb1490 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:14:47.932646 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 769/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 06:14:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:14:48.214800 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9184c9-7111-484f-b03a-1f166d77a347 None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:14:48.226931 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9184c9-7111-484f-b03a-1f166d77a347 None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:14:48.230398 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb136e5-a8d0-4b95-9188-777e4c4de20f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bb6c79db75e4d169331c02133d1ec32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:14:48.230989 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb136e5-a8d0-4b95-9188-777e4c4de20f service neutron] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:14:48.231267 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb136e5-a8d0-4b95-9188-777e4c4de20f service neutron] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:14:48.231536 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb136e5-a8d0-4b95-9188-777e4c4de20f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:14:48.240615 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb136e5-a8d0-4b95-9188-777e4c4de20f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:14:48.242031 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb136e5-a8d0-4b95-9188-777e4c4de20f service neutron] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:14:48.251389 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 764/1533] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:14:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:14:51.719497 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccb70b9-5b5c-4568-9998-2ee1503caf23 None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:14:51.719974 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccb70b9-5b5c-4568-9998-2ee1503caf23 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:14:51.720079 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccb70b9-5b5c-4568-9998-2ee1503caf23 None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:14:51.721490 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 770/1534] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 06:14:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:14:51.727012 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3dd4f7-9b40-418c-ba59-ef82ed1d4537 None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:14:51.727409 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3dd4f7-9b40-418c-ba59-ef82ed1d4537 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:14:51.727563 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3dd4f7-9b40-418c-ba59-ef82ed1d4537 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:14:51.746987 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-ea3dd4f7-9b40-418c-ba59-ef82ed1d4537 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:14:51.748173 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-ea3dd4f7-9b40-418c-ba59-ef82ed1d4537 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:14:51.756929 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 765/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 06:14:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:14:52.311741 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccb70b9-5b5c-4568-9998-2ee1503caf23 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:14:52.313507 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-5ccb70b9-5b5c-4568-9998-2ee1503caf23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:14:52.379875 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:14:52.394592 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccb70b9-5b5c-4568-9998-2ee1503caf23 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:14:52.398892 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b63985-ca0b-494e-bad8-42e813024160 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c4def55ac0e48b29836bc5c64bc7402', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:14:52.398892 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b63985-ca0b-494e-bad8-42e813024160 service cinder] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:14:52.399225 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b63985-ca0b-494e-bad8-42e813024160 service cinder] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:14:52.399645 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b63985-ca0b-494e-bad8-42e813024160 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:14:52.412812 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b63985-ca0b-494e-bad8-42e813024160 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:14:52.414246 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b63985-ca0b-494e-bad8-42e813024160 service cinder] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:14:52.423725 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 771/1536] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:14:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:14:54.372660 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10833cab-fb1c-42c7-a339-11be78c5698b None None] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:14:54.373288 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10833cab-fb1c-42c7-a339-11be78c5698b None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:14:54.373288 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10833cab-fb1c-42c7-a339-11be78c5698b None None] PATH_INFO: `/` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:14:54.374511 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 766/1537] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 06:14:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:14:54.380298 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dc7971-17d9-4542-bae9-f3c55450d544 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:14:54.380782 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dc7971-17d9-4542-bae9-f3c55450d544 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:14:54.380887 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dc7971-17d9-4542-bae9-f3c55450d544 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:14:54.399960 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-45dc7971-17d9-4542-bae9-f3c55450d544 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:14:54.401190 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-45dc7971-17d9-4542-bae9-f3c55450d544 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:14:54.409495 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 772/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 06:14:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:14:54.962012 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10833cab-fb1c-42c7-a339-11be78c5698b None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:14:54.963709 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-10833cab-fb1c-42c7-a339-11be78c5698b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:14:54.977005 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10833cab-fb1c-42c7-a339-11be78c5698b None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:14:54.980437 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c25cfd8-5725-47bc-b5e4-2b7b1dd29535 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c4def55ac0e48b29836bc5c64bc7402', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:14:54.981047 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c25cfd8-5725-47bc-b5e4-2b7b1dd29535 service cinder] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:14:54.981323 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c25cfd8-5725-47bc-b5e4-2b7b1dd29535 service cinder] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:14:54.981611 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c25cfd8-5725-47bc-b5e4-2b7b1dd29535 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:14:54.990854 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c25cfd8-5725-47bc-b5e4-2b7b1dd29535 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:14:54.992252 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c25cfd8-5725-47bc-b5e4-2b7b1dd29535 service cinder] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:14:55.001551 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 767/1539] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:14:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:15:04.395252 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=11 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:15:04.997556 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=89 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:16:19.789282 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e792110-634d-41c0-90c9-c4f1d21d10fa None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:19.789936 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e792110-634d-41c0-90c9-c4f1d21d10fa None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:19.789936 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e792110-634d-41c0-90c9-c4f1d21d10fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:19.820752 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-2e792110-634d-41c0-90c9-c4f1d21d10fa None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:16:19.821885 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-2e792110-634d-41c0-90c9-c4f1d21d10fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:16:19.835406 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 773/1540] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:16:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:16:19.845219 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5500275d-3bb9-41f8-9bf2-71f681b8fe5e None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:19.846203 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5500275d-3bb9-41f8-9bf2-71f681b8fe5e None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:19.846698 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5500275d-3bb9-41f8-9bf2-71f681b8fe5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:19.877689 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-5500275d-3bb9-41f8-9bf2-71f681b8fe5e None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:16:19.879452 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-5500275d-3bb9-41f8-9bf2-71f681b8fe5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:16:19.894942 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 768/1541] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:16:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:16:19.903787 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e792110-634d-41c0-90c9-c4f1d21d10fa None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:16:19.917602 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e792110-634d-41c0-90c9-c4f1d21d10fa None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:16:19.927606 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb38d7ea-f507-4b42-80d8-7f36ae2a2e67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:16:19.928246 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb38d7ea-f507-4b42-80d8-7f36ae2a2e67 admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:19.928543 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb38d7ea-f507-4b42-80d8-7f36ae2a2e67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:19.934823 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb38d7ea-f507-4b42-80d8-7f36ae2a2e67 admin admin] PATH_INFO: `/v3/domains` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:19.934823 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb38d7ea-f507-4b42-80d8-7f36ae2a2e67 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:16:19.934823 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb38d7ea-f507-4b42-80d8-7f36ae2a2e67 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:16:19.934823 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb38d7ea-f507-4b42-80d8-7f36ae2a2e67 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:16:19.948656 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 774/1542] 10.222.0.62 () {58 vars in 1153 bytes} [Wed Aug 24 06:16:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:16:19.959410 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5500275d-3bb9-41f8-9bf2-71f681b8fe5e None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:16:19.973734 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5500275d-3bb9-41f8-9bf2-71f681b8fe5e None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:16:19.987104 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79804b60-3b81-4f72-b6dc-82e55c8afd33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:16:19.988161 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79804b60-3b81-4f72-b6dc-82e55c8afd33 admin admin] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:19.989268 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79804b60-3b81-4f72-b6dc-82e55c8afd33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:19.989882 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79804b60-3b81-4f72-b6dc-82e55c8afd33 admin admin] PATH_INFO: `/v3/projects` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:19.991382 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79804b60-3b81-4f72-b6dc-82e55c8afd33 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:16:19.993573 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79804b60-3b81-4f72-b6dc-82e55c8afd33 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:16:20.136979 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 769/1543] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Aug 24 06:16:19 2022] POST /identity/v3/projects => generated 377 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:16:20.141686 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb38d7ea-f507-4b42-80d8-7f36ae2a2e67 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:16:20.158495 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb38d7ea-f507-4b42-80d8-7f36ae2a2e67 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:16:20.245570 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d6eb31-472a-4994-ab29-b2bba3ee7907 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:16:20.246133 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d6eb31-472a-4994-ab29-b2bba3ee7907 admin admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:20.246398 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d6eb31-472a-4994-ab29-b2bba3ee7907 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:20.246674 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d6eb31-472a-4994-ab29-b2bba3ee7907 admin admin] PATH_INFO: `/v3/users` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:20.247304 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d6eb31-472a-4994-ab29-b2bba3ee7907 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:16:20.249546 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d6eb31-472a-4994-ab29-b2bba3ee7907 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:16:20.358567 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 775/1544] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Aug 24 06:16:20 2022] POST /identity/v3/users => generated 289 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:16:20.363218 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79804b60-3b81-4f72-b6dc-82e55c8afd33 admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:16:20.375261 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79804b60-3b81-4f72-b6dc-82e55c8afd33 admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:16:20.385505 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c5e1e7-936c-477d-a963-8f6e375b7884 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:16:20.386167 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c5e1e7-936c-477d-a963-8f6e375b7884 admin admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:20.386453 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c5e1e7-936c-477d-a963-8f6e375b7884 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:20.386727 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c5e1e7-936c-477d-a963-8f6e375b7884 admin admin] PATH_INFO: `/v3/roles` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:20.387397 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c5e1e7-936c-477d-a963-8f6e375b7884 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:16:20.389116 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c5e1e7-936c-477d-a963-8f6e375b7884 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:16:20.405272 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 770/1545] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Aug 24 06:16:20 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:16:20.409354 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d6eb31-472a-4994-ab29-b2bba3ee7907 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:16:20.427015 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d6eb31-472a-4994-ab29-b2bba3ee7907 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:16:20.432020 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b27145d-bd88-4b65-9528-d590abf1c3b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:16:20.432701 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b27145d-bd88-4b65-9528-d590abf1c3b0 admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:20.432986 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b27145d-bd88-4b65-9528-d590abf1c3b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:20.433264 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b27145d-bd88-4b65-9528-d590abf1c3b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:20.434248 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b27145d-bd88-4b65-9528-d590abf1c3b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:16:20.435929 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b27145d-bd88-4b65-9528-d590abf1c3b0 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:16:20.442055 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4783966-b406-4299-9c58-963ade763ca7 None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:20.442403 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4783966-b406-4299-9c58-963ade763ca7 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:20.442604 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4783966-b406-4299-9c58-963ade763ca7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:20.466952 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 776/1546] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Aug 24 06:16:20 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:16:20.471075 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b27145d-bd88-4b65-9528-d590abf1c3b0 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:16:20.476799 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-c4783966-b406-4299-9c58-963ade763ca7 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:16:20.478015 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-c4783966-b406-4299-9c58-963ade763ca7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:16:20.484250 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b27145d-bd88-4b65-9528-d590abf1c3b0 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:16:20.499430 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 771/1547] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:16:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 06:16:20.502756 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ea1543-595f-4678-85f2-7ca24f1a7845 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:16:20.503885 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ea1543-595f-4678-85f2-7ca24f1a7845 admin admin] REQUEST_METHOD: `PUT` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:20.504931 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ea1543-595f-4678-85f2-7ca24f1a7845 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:20.505436 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ea1543-595f-4678-85f2-7ca24f1a7845 admin admin] PATH_INFO: `/v3/projects/ff194aa4a2d744caaa09e0ceee54b1d2/users/25908fb02f7f46f5b9e483811302a67e/roles/580aa7dbb38d4cff8f9d1bb8983f99ef` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:20.509073 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc7adbf-332e-422b-b53b-a01c2c1bbd2e None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:20.509363 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc7adbf-332e-422b-b53b-a01c2c1bbd2e None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:20.509642 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc7adbf-332e-422b-b53b-a01c2c1bbd2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:20.534315 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ea1543-595f-4678-85f2-7ca24f1a7845 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ff194aa4a2d744caaa09e0ceee54b1d2, user_id=25908fb02f7f46f5b9e483811302a67e, role_id=580aa7dbb38d4cff8f9d1bb8983f99ef)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:16:20.535922 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: WARNING py.warnings [None req-f9ea1543-595f-4678-85f2-7ca24f1a7845 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 06:16:20.535922 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: warnings.warn(msg) Aug 24 06:16:20.535922 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: Aug 24 06:16:20.538633 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ea1543-595f-4678-85f2-7ca24f1a7845 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:16:20.548636 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-2bc7adbf-332e-422b-b53b-a01c2c1bbd2e None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:16:20.550079 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-2bc7adbf-332e-422b-b53b-a01c2c1bbd2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:16:20.567944 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 772/1548] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:16:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 06:16:20.569094 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1b1745-cf26-4da3-9db6-b53011cec861 None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:20.569378 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1b1745-cf26-4da3-9db6-b53011cec861 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:20.569650 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1b1745-cf26-4da3-9db6-b53011cec861 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:20.601957 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-0f1b1745-cf26-4da3-9db6-b53011cec861 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:16:20.603164 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-0f1b1745-cf26-4da3-9db6-b53011cec861 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:16:20.618203 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 773/1549] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:16:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:16:20.619057 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1b1745-cf26-4da3-9db6-b53011cec861 None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:16:20.632119 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1b1745-cf26-4da3-9db6-b53011cec861 None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:16:20.638052 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8103d2e-23b4-49ef-928c-4badc2c2d8fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:16:20.644626 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8103d2e-23b4-49ef-928c-4badc2c2d8fa admin admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:20.644626 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8103d2e-23b4-49ef-928c-4badc2c2d8fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:20.644626 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8103d2e-23b4-49ef-928c-4badc2c2d8fa admin admin] PATH_INFO: `/v3/domains` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:20.644626 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8103d2e-23b4-49ef-928c-4badc2c2d8fa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:16:20.644626 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8103d2e-23b4-49ef-928c-4badc2c2d8fa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:16:20.645100 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: WARNING py.warnings [None req-e8103d2e-23b4-49ef-928c-4badc2c2d8fa admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 24 06:16:20.645100 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: warnings.warn(msg) Aug 24 06:16:20.645100 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: Aug 24 06:16:20.645100 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8103d2e-23b4-49ef-928c-4badc2c2d8fa admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:16:20.648300 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 777/1550] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Aug 24 06:16:20 2022] PUT /identity/v3/projects/ff194aa4a2d744caaa09e0ceee54b1d2/users/25908fb02f7f46f5b9e483811302a67e/roles/580aa7dbb38d4cff8f9d1bb8983f99ef => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:16:20.652335 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2105501-3b05-4d2a-b321-b16438bed2d1 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:20.652921 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2105501-3b05-4d2a-b321-b16438bed2d1 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:20.653425 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2105501-3b05-4d2a-b321-b16438bed2d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:20.663413 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 774/1551] 10.222.0.62 () {58 vars in 1153 bytes} [Wed Aug 24 06:16:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:16:20.664824 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314c6e60-1e69-45f3-8350-c0ef287e2e47 None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:20.665110 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314c6e60-1e69-45f3-8350-c0ef287e2e47 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:20.665388 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314c6e60-1e69-45f3-8350-c0ef287e2e47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:20.697809 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-e2105501-3b05-4d2a-b321-b16438bed2d1 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:16:20.700942 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-e2105501-3b05-4d2a-b321-b16438bed2d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:16:20.705854 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-314c6e60-1e69-45f3-8350-c0ef287e2e47 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:16:20.713690 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-314c6e60-1e69-45f3-8350-c0ef287e2e47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:16:20.789064 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 778/1552] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:16:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:16:20.789738 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 775/1553] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:16:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:16:20.790583 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2105501-3b05-4d2a-b321-b16438bed2d1 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:16:20.791087 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb8c192-01a1-4c86-8044-99e63d884d02 None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:20.791342 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb8c192-01a1-4c86-8044-99e63d884d02 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:20.791617 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb8c192-01a1-4c86-8044-99e63d884d02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:20.803443 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2105501-3b05-4d2a-b321-b16438bed2d1 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:16:20.808269 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1328c76-91bf-4cbe-9044-34733d832b8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:16:20.808914 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1328c76-91bf-4cbe-9044-34733d832b8c admin admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:20.809339 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1328c76-91bf-4cbe-9044-34733d832b8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:20.809473 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1328c76-91bf-4cbe-9044-34733d832b8c admin admin] PATH_INFO: `/v3/projects` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:20.810078 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1328c76-91bf-4cbe-9044-34733d832b8c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:16:20.811120 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: WARNING py.warnings [None req-b1328c76-91bf-4cbe-9044-34733d832b8c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 06:16:20.811120 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: warnings.warn(msg) Aug 24 06:16:20.811120 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: Aug 24 06:16:20.811675 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-eeb8c192-01a1-4c86-8044-99e63d884d02 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:16:20.812318 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1328c76-91bf-4cbe-9044-34733d832b8c admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:16:20.812952 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-eeb8c192-01a1-4c86-8044-99e63d884d02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:16:20.821059 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 776/1554] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:16:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:16:20.822258 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5408367a-ccb3-4fd6-a528-805f86cc8755 None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:20.822544 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5408367a-ccb3-4fd6-a528-805f86cc8755 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:20.822820 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5408367a-ccb3-4fd6-a528-805f86cc8755 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:20.842347 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-5408367a-ccb3-4fd6-a528-805f86cc8755 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:16:20.843654 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-5408367a-ccb3-4fd6-a528-805f86cc8755 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:16:20.852144 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 777/1555] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:16:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:16:20.853365 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc37394a-5a87-4a1e-a8a4-d21662daf257 None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:20.853647 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc37394a-5a87-4a1e-a8a4-d21662daf257 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:20.853920 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc37394a-5a87-4a1e-a8a4-d21662daf257 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:20.873178 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-bc37394a-5a87-4a1e-a8a4-d21662daf257 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:16:20.874342 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-bc37394a-5a87-4a1e-a8a4-d21662daf257 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:16:20.882652 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 778/1556] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:16:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:16:20.883615 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624384e0-6ffa-43ab-959f-2b02845e7a75 None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:20.883775 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 779/1557] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Aug 24 06:16:20 2022] POST /identity/v3/projects => generated 403 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:16:20.883886 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624384e0-6ffa-43ab-959f-2b02845e7a75 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:20.884159 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624384e0-6ffa-43ab-959f-2b02845e7a75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:20.885059 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f439c8f-e00c-44e5-9a1c-f9c8530edbf8 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:20.889172 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f439c8f-e00c-44e5-9a1c-f9c8530edbf8 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:20.889172 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f439c8f-e00c-44e5-9a1c-f9c8530edbf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:20.904872 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-624384e0-6ffa-43ab-959f-2b02845e7a75 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:16:20.906069 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-624384e0-6ffa-43ab-959f-2b02845e7a75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:16:20.906625 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-9f439c8f-e00c-44e5-9a1c-f9c8530edbf8 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:16:20.907811 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-9f439c8f-e00c-44e5-9a1c-f9c8530edbf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:16:20.917395 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:16:20.956443 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:16:20.965570 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 779/1558] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:16:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:16:20.966382 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624384e0-6ffa-43ab-959f-2b02845e7a75 None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:16:20.975503 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 780/1559] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:16:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:16:20.976018 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624384e0-6ffa-43ab-959f-2b02845e7a75 None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:16:20.976955 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f439c8f-e00c-44e5-9a1c-f9c8530edbf8 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:16:20.985609 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af0cd2b-d33f-4005-b5ab-a3c745eae933 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:16:20.986251 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af0cd2b-d33f-4005-b5ab-a3c745eae933 admin admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:20.986561 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af0cd2b-d33f-4005-b5ab-a3c745eae933 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:20.986835 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af0cd2b-d33f-4005-b5ab-a3c745eae933 admin admin] PATH_INFO: `/v3/domains` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:20.987515 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af0cd2b-d33f-4005-b5ab-a3c745eae933 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:16:20.987835 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af0cd2b-d33f-4005-b5ab-a3c745eae933 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:16:20.990044 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af0cd2b-d33f-4005-b5ab-a3c745eae933 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:16:21.002619 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 780/1560] 10.222.0.62 () {58 vars in 1153 bytes} [Wed Aug 24 06:16:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:16:21.003863 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce2f6d7-d61a-4d1b-9fe4-f3526fac3cb9 None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:21.004459 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce2f6d7-d61a-4d1b-9fe4-f3526fac3cb9 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:21.004459 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce2f6d7-d61a-4d1b-9fe4-f3526fac3cb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:21.023736 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-bce2f6d7-d61a-4d1b-9fe4-f3526fac3cb9 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:16:21.025011 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-bce2f6d7-d61a-4d1b-9fe4-f3526fac3cb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:16:21.032800 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 781/1561] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:16:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:16:21.033984 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f735a2-67c8-49f2-99e4-215e7e6323fd None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:21.034480 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f735a2-67c8-49f2-99e4-215e7e6323fd None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:21.034924 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f735a2-67c8-49f2-99e4-215e7e6323fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:21.038726 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f439c8f-e00c-44e5-9a1c-f9c8530edbf8 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:16:21.042410 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fac127d-1ff5-41ba-be16-217082ca6f53 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bb6c79db75e4d169331c02133d1ec32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:16:21.042995 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fac127d-1ff5-41ba-be16-217082ca6f53 service neutron] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:21.043276 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fac127d-1ff5-41ba-be16-217082ca6f53 service neutron] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:21.043547 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fac127d-1ff5-41ba-be16-217082ca6f53 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:21.053183 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fac127d-1ff5-41ba-be16-217082ca6f53 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:16:21.055921 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-d8f735a2-67c8-49f2-99e4-215e7e6323fd None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:16:21.056035 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fac127d-1ff5-41ba-be16-217082ca6f53 service neutron] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:16:21.056139 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-d8f735a2-67c8-49f2-99e4-215e7e6323fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:16:21.064106 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 782/1562] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:16:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:16:21.064479 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 781/1563] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:16:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:16:21.065132 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0ef386-d3ee-47b5-b038-a48326d1d12c None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:21.065378 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fac127d-1ff5-41ba-be16-217082ca6f53 service neutron] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:16:21.065516 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0ef386-d3ee-47b5-b038-a48326d1d12c None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:21.065699 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0ef386-d3ee-47b5-b038-a48326d1d12c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:21.079510 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fac127d-1ff5-41ba-be16-217082ca6f53 service neutron] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:16:21.085260 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4be26a6-a9cc-43dd-87fe-9d7caa72c7e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:16:21.085836 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-cb0ef386-d3ee-47b5-b038-a48326d1d12c None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:16:21.085968 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4be26a6-a9cc-43dd-87fe-9d7caa72c7e7 admin admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:21.086148 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4be26a6-a9cc-43dd-87fe-9d7caa72c7e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:21.086420 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4be26a6-a9cc-43dd-87fe-9d7caa72c7e7 admin admin] PATH_INFO: `/v3/users` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:21.087089 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4be26a6-a9cc-43dd-87fe-9d7caa72c7e7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:16:21.087215 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-cb0ef386-d3ee-47b5-b038-a48326d1d12c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:16:21.089088 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4be26a6-a9cc-43dd-87fe-9d7caa72c7e7 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:16:21.095309 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 783/1564] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:16:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:16:21.096513 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6534df63-59ea-4c87-8742-abac0ddf290a None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:21.096806 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6534df63-59ea-4c87-8742-abac0ddf290a None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:21.097081 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6534df63-59ea-4c87-8742-abac0ddf290a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:21.116929 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-6534df63-59ea-4c87-8742-abac0ddf290a None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:16:21.118095 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-6534df63-59ea-4c87-8742-abac0ddf290a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:16:21.125830 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 784/1565] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:16:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:16:21.126952 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953d92e7-2905-44b4-930d-33c24b294146 None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:21.127229 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953d92e7-2905-44b4-930d-33c24b294146 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:21.127498 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953d92e7-2905-44b4-930d-33c24b294146 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:21.145968 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-953d92e7-2905-44b4-930d-33c24b294146 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:16:21.147135 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-953d92e7-2905-44b4-930d-33c24b294146 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:16:21.154802 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 785/1566] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:16:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:16:21.156142 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953d92e7-2905-44b4-930d-33c24b294146 None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:16:21.165733 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953d92e7-2905-44b4-930d-33c24b294146 None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:16:21.170199 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2d8c56-c024-4f0a-9b88-63bccc43ce72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:16:21.170786 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2d8c56-c024-4f0a-9b88-63bccc43ce72 admin admin] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:21.171063 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2d8c56-c024-4f0a-9b88-63bccc43ce72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:21.171356 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2d8c56-c024-4f0a-9b88-63bccc43ce72 admin admin] PATH_INFO: `/v3/projects` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:21.171951 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2d8c56-c024-4f0a-9b88-63bccc43ce72 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:16:21.173857 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2d8c56-c024-4f0a-9b88-63bccc43ce72 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:16:21.239964 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 782/1567] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Aug 24 06:16:21 2022] POST /identity/v3/users => generated 302 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:16:21.239964 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4be26a6-a9cc-43dd-87fe-9d7caa72c7e7 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:16:21.251944 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4be26a6-a9cc-43dd-87fe-9d7caa72c7e7 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:16:21.263155 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2672f33-2a3f-44ee-9d33-17784300e137 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:16:21.263891 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2672f33-2a3f-44ee-9d33-17784300e137 admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:21.264299 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2672f33-2a3f-44ee-9d33-17784300e137 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:21.264701 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2672f33-2a3f-44ee-9d33-17784300e137 admin admin] PATH_INFO: `/v3/domains` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:21.265417 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2672f33-2a3f-44ee-9d33-17784300e137 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:16:21.265793 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2672f33-2a3f-44ee-9d33-17784300e137 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:16:21.267462 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2672f33-2a3f-44ee-9d33-17784300e137 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:16:21.268352 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 786/1568] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Aug 24 06:16:21 2022] POST /identity/v3/projects => generated 377 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:16:21.273235 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2d8c56-c024-4f0a-9b88-63bccc43ce72 admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:16:21.289783 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 783/1569] 10.222.0.62 () {58 vars in 1153 bytes} [Wed Aug 24 06:16:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:16:21.289783 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2672f33-2a3f-44ee-9d33-17784300e137 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:16:21.298319 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2672f33-2a3f-44ee-9d33-17784300e137 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:16:21.345257 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2d8c56-c024-4f0a-9b88-63bccc43ce72 admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:16:21.351775 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b838cc03-c39f-4cf3-a3c2-4bff093f314e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:16:21.352393 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b838cc03-c39f-4cf3-a3c2-4bff093f314e admin admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:21.352692 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b838cc03-c39f-4cf3-a3c2-4bff093f314e admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:21.352963 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b838cc03-c39f-4cf3-a3c2-4bff093f314e admin admin] PATH_INFO: `/v3/domains` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:21.353612 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b838cc03-c39f-4cf3-a3c2-4bff093f314e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:16:21.353914 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b838cc03-c39f-4cf3-a3c2-4bff093f314e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:16:21.355568 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b838cc03-c39f-4cf3-a3c2-4bff093f314e admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:16:21.356839 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefc2ece-2d2e-4847-af42-4b9042c25c9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:16:21.357339 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefc2ece-2d2e-4847-af42-4b9042c25c9a admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:21.357904 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefc2ece-2d2e-4847-af42-4b9042c25c9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:21.358658 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefc2ece-2d2e-4847-af42-4b9042c25c9a admin admin] PATH_INFO: `/v3/domains` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:21.359487 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefc2ece-2d2e-4847-af42-4b9042c25c9a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:16:21.359810 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefc2ece-2d2e-4847-af42-4b9042c25c9a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:16:21.361663 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefc2ece-2d2e-4847-af42-4b9042c25c9a admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:16:21.368277 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 787/1570] 10.222.0.62 () {58 vars in 1153 bytes} [Wed Aug 24 06:16:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:16:21.369017 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b838cc03-c39f-4cf3-a3c2-4bff093f314e admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:16:21.375425 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 784/1571] 10.222.0.62 () {58 vars in 1153 bytes} [Wed Aug 24 06:16:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:16:21.385646 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefc2ece-2d2e-4847-af42-4b9042c25c9a admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:16:21.385810 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b838cc03-c39f-4cf3-a3c2-4bff093f314e admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:16:21.387282 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefc2ece-2d2e-4847-af42-4b9042c25c9a admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:16:21.391819 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420b3d46-657c-4030-a24b-c7837ddda7ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:16:21.392448 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420b3d46-657c-4030-a24b-c7837ddda7ba admin admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:21.392761 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420b3d46-657c-4030-a24b-c7837ddda7ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:21.393024 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420b3d46-657c-4030-a24b-c7837ddda7ba admin admin] PATH_INFO: `/v3/domains` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:21.393872 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420b3d46-657c-4030-a24b-c7837ddda7ba admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:16:21.394099 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420b3d46-657c-4030-a24b-c7837ddda7ba admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:16:21.394418 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58faf198-6c1a-4dec-b03c-d6ab3959c0b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:16:21.395757 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58faf198-6c1a-4dec-b03c-d6ab3959c0b4 admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:21.396189 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58faf198-6c1a-4dec-b03c-d6ab3959c0b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:21.396428 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420b3d46-657c-4030-a24b-c7837ddda7ba admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:16:21.396935 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58faf198-6c1a-4dec-b03c-d6ab3959c0b4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:21.397731 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58faf198-6c1a-4dec-b03c-d6ab3959c0b4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:16:21.398191 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58faf198-6c1a-4dec-b03c-d6ab3959c0b4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:16:21.399921 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58faf198-6c1a-4dec-b03c-d6ab3959c0b4 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:16:21.410003 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 788/1572] 10.222.0.62 () {58 vars in 1153 bytes} [Wed Aug 24 06:16:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:16:21.411041 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420b3d46-657c-4030-a24b-c7837ddda7ba admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:16:21.413602 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 785/1573] 10.222.0.62 () {58 vars in 1153 bytes} [Wed Aug 24 06:16:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:16:21.414787 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58faf198-6c1a-4dec-b03c-d6ab3959c0b4 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:16:21.429284 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420b3d46-657c-4030-a24b-c7837ddda7ba admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:16:21.434562 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb92e238-bdbf-42e7-8870-901e7bf34069 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:16:21.435408 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb92e238-bdbf-42e7-8870-901e7bf34069 admin admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:21.435623 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb92e238-bdbf-42e7-8870-901e7bf34069 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:21.435898 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb92e238-bdbf-42e7-8870-901e7bf34069 admin admin] PATH_INFO: `/v3/roles` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:21.436018 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58faf198-6c1a-4dec-b03c-d6ab3959c0b4 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:16:21.436641 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb92e238-bdbf-42e7-8870-901e7bf34069 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:16:21.438300 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb92e238-bdbf-42e7-8870-901e7bf34069 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:16:21.442651 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287d1a73-4453-47fc-adaa-99d73f358b37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:16:21.443171 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287d1a73-4453-47fc-adaa-99d73f358b37 admin admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:21.443580 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287d1a73-4453-47fc-adaa-99d73f358b37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:21.444075 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287d1a73-4453-47fc-adaa-99d73f358b37 admin admin] PATH_INFO: `/v3/users` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:21.445085 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287d1a73-4453-47fc-adaa-99d73f358b37 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:16:21.448696 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287d1a73-4453-47fc-adaa-99d73f358b37 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:16:21.450535 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 789/1574] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Aug 24 06:16:21 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:16:21.451500 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb92e238-bdbf-42e7-8870-901e7bf34069 admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:16:21.460857 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:16:21.465691 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb92e238-bdbf-42e7-8870-901e7bf34069 admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:16:21.470389 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc7958e-7f8c-4767-bde3-8ba248022af2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:16:21.470987 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc7958e-7f8c-4767-bde3-8ba248022af2 admin admin] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:21.471261 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc7958e-7f8c-4767-bde3-8ba248022af2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:21.471528 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc7958e-7f8c-4767-bde3-8ba248022af2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:21.472144 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc7958e-7f8c-4767-bde3-8ba248022af2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:16:21.474100 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc7958e-7f8c-4767-bde3-8ba248022af2 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:16:21.554429 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 786/1575] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Aug 24 06:16:21 2022] POST /identity/v3/users => generated 289 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:16:21.554429 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287d1a73-4453-47fc-adaa-99d73f358b37 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:16:21.554741 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 790/1576] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Aug 24 06:16:21 2022] POST /identity/v3/projects => generated 373 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:16:21.554741 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc7958e-7f8c-4767-bde3-8ba248022af2 admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:16:21.562080 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287d1a73-4453-47fc-adaa-99d73f358b37 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:16:21.602924 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:16:21.628355 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc7958e-7f8c-4767-bde3-8ba248022af2 admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:16:21.628813 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed899af4-bbd5-4b34-b330-6c1482ac4ff3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:16:21.630017 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed899af4-bbd5-4b34-b330-6c1482ac4ff3 admin admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:21.630317 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed899af4-bbd5-4b34-b330-6c1482ac4ff3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:21.630631 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed899af4-bbd5-4b34-b330-6c1482ac4ff3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:21.631237 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed899af4-bbd5-4b34-b330-6c1482ac4ff3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:16:21.633413 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed899af4-bbd5-4b34-b330-6c1482ac4ff3 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:16:21.635602 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63f5647-8511-48e9-b1f3-748714ebdf02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:16:21.636405 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63f5647-8511-48e9-b1f3-748714ebdf02 admin admin] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:21.636919 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63f5647-8511-48e9-b1f3-748714ebdf02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:21.637404 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63f5647-8511-48e9-b1f3-748714ebdf02 admin admin] PATH_INFO: `/v3/projects` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:21.638426 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63f5647-8511-48e9-b1f3-748714ebdf02 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:16:21.640812 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63f5647-8511-48e9-b1f3-748714ebdf02 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:16:21.742113 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 787/1577] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Aug 24 06:16:21 2022] POST /identity/v3/projects => generated 403 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:16:21.742756 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed899af4-bbd5-4b34-b330-6c1482ac4ff3 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:16:21.746972 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 791/1578] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Aug 24 06:16:21 2022] POST /identity/v3/projects => generated 381 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:16:21.747839 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63f5647-8511-48e9-b1f3-748714ebdf02 admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:16:21.808255 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed899af4-bbd5-4b34-b330-6c1482ac4ff3 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:16:21.811197 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63f5647-8511-48e9-b1f3-748714ebdf02 admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:16:21.813406 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd905f0-06e5-4371-836c-8913165f6237 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:16:21.814019 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd905f0-06e5-4371-836c-8913165f6237 admin admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:21.814296 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd905f0-06e5-4371-836c-8913165f6237 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:21.814564 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd905f0-06e5-4371-836c-8913165f6237 admin admin] PATH_INFO: `/v3/projects` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:21.815186 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd905f0-06e5-4371-836c-8913165f6237 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:16:21.817201 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb8bb3e-a84d-446a-85c0-33e78c660fae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:16:21.817767 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd905f0-06e5-4371-836c-8913165f6237 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:16:21.817877 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb8bb3e-a84d-446a-85c0-33e78c660fae admin admin] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:21.818087 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb8bb3e-a84d-446a-85c0-33e78c660fae admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:21.818361 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb8bb3e-a84d-446a-85c0-33e78c660fae admin admin] PATH_INFO: `/v3/projects` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:21.818980 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb8bb3e-a84d-446a-85c0-33e78c660fae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:16:21.820923 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb8bb3e-a84d-446a-85c0-33e78c660fae admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:16:21.909872 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 788/1579] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Aug 24 06:16:21 2022] POST /identity/v3/projects => generated 387 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:16:21.909872 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd905f0-06e5-4371-836c-8913165f6237 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:16:21.937577 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 792/1580] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Aug 24 06:16:21 2022] POST /identity/v3/projects => generated 381 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:16:21.937577 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb8bb3e-a84d-446a-85c0-33e78c660fae admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:16:21.983746 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd905f0-06e5-4371-836c-8913165f6237 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:16:21.988032 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:16:22.005927 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb8bb3e-a84d-446a-85c0-33e78c660fae admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:16:22.010626 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82633273-ab1b-4ee1-8c8c-ee61aa26bd0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:16:22.014763 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82633273-ab1b-4ee1-8c8c-ee61aa26bd0b admin admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:22.014763 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82633273-ab1b-4ee1-8c8c-ee61aa26bd0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:22.014763 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82633273-ab1b-4ee1-8c8c-ee61aa26bd0b admin admin] PATH_INFO: `/v3/roles` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:22.014763 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82633273-ab1b-4ee1-8c8c-ee61aa26bd0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:16:22.014763 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82633273-ab1b-4ee1-8c8c-ee61aa26bd0b admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:16:22.024536 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 793/1581] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Aug 24 06:16:21 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:16:22.028409 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82633273-ab1b-4ee1-8c8c-ee61aa26bd0b admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:16:22.039834 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82633273-ab1b-4ee1-8c8c-ee61aa26bd0b admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:16:22.046711 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b5c9cc-8395-4fbb-92e0-7ba141a96c54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:16:22.046711 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b5c9cc-8395-4fbb-92e0-7ba141a96c54 admin admin] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:22.046711 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b5c9cc-8395-4fbb-92e0-7ba141a96c54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:22.046711 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b5c9cc-8395-4fbb-92e0-7ba141a96c54 admin admin] PATH_INFO: `/v3/users` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:22.047127 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98e386d-1a96-4133-ac75-130b5dc3df3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:16:22.047242 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b5c9cc-8395-4fbb-92e0-7ba141a96c54 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:16:22.047348 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98e386d-1a96-4133-ac75-130b5dc3df3b admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:22.047517 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98e386d-1a96-4133-ac75-130b5dc3df3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:22.047671 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: WARNING py.warnings [None req-78b5c9cc-8395-4fbb-92e0-7ba141a96c54 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 06:16:22.047671 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: warnings.warn(msg) Aug 24 06:16:22.047671 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: Aug 24 06:16:22.047934 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98e386d-1a96-4133-ac75-130b5dc3df3b admin admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:22.048487 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98e386d-1a96-4133-ac75-130b5dc3df3b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:16:22.048774 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b5c9cc-8395-4fbb-92e0-7ba141a96c54 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:16:22.050369 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98e386d-1a96-4133-ac75-130b5dc3df3b admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:16:22.061524 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 789/1582] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Aug 24 06:16:21 2022] GET /identity/v3/roles => generated 1420 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:16:22.062331 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98e386d-1a96-4133-ac75-130b5dc3df3b admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:16:22.074910 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98e386d-1a96-4133-ac75-130b5dc3df3b admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:16:22.077448 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c20532-47a9-4026-8832-be7b16f2e7c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:16:22.078360 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c20532-47a9-4026-8832-be7b16f2e7c2 admin admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:22.078777 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c20532-47a9-4026-8832-be7b16f2e7c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:22.078910 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c20532-47a9-4026-8832-be7b16f2e7c2 admin admin] PATH_INFO: `/v3/users` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:22.079518 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c20532-47a9-4026-8832-be7b16f2e7c2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:16:22.081512 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c20532-47a9-4026-8832-be7b16f2e7c2 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:16:22.160061 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 794/1583] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Aug 24 06:16:22 2022] POST /identity/v3/users => generated 287 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:16:22.161082 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b5c9cc-8395-4fbb-92e0-7ba141a96c54 admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:16:22.170631 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b5c9cc-8395-4fbb-92e0-7ba141a96c54 admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:16:22.175409 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b2c2fe-71f4-4791-a187-9c56ccc7ee87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:16:22.176013 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b2c2fe-71f4-4791-a187-9c56ccc7ee87 admin admin] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:22.176284 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b2c2fe-71f4-4791-a187-9c56ccc7ee87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:22.176584 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b2c2fe-71f4-4791-a187-9c56ccc7ee87 admin admin] PATH_INFO: `/v3/users` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:22.177219 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b2c2fe-71f4-4791-a187-9c56ccc7ee87 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:16:22.179151 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b2c2fe-71f4-4791-a187-9c56ccc7ee87 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:16:22.188332 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 790/1584] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Aug 24 06:16:22 2022] POST /identity/v3/users => generated 302 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:16:22.188332 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c20532-47a9-4026-8832-be7b16f2e7c2 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:16:22.196758 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c20532-47a9-4026-8832-be7b16f2e7c2 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:16:22.201516 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128896c1-1de3-4339-881d-30cffb273c64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:16:22.202100 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128896c1-1de3-4339-881d-30cffb273c64 admin admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:22.202377 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128896c1-1de3-4339-881d-30cffb273c64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:22.202651 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128896c1-1de3-4339-881d-30cffb273c64 admin admin] PATH_INFO: `/v3/users` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:22.203261 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128896c1-1de3-4339-881d-30cffb273c64 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:16:22.205364 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128896c1-1de3-4339-881d-30cffb273c64 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:16:22.271693 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 795/1585] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Aug 24 06:16:22 2022] POST /identity/v3/users => generated 291 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:16:22.275604 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b2c2fe-71f4-4791-a187-9c56ccc7ee87 admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:16:22.282086 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b2c2fe-71f4-4791-a187-9c56ccc7ee87 admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:16:22.286648 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da00536-1428-4cd3-b08c-b4b72b051df3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:16:22.287228 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da00536-1428-4cd3-b08c-b4b72b051df3 admin admin] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:22.287501 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da00536-1428-4cd3-b08c-b4b72b051df3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:22.287770 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da00536-1428-4cd3-b08c-b4b72b051df3 admin admin] PATH_INFO: `/v3/users` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:22.288379 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da00536-1428-4cd3-b08c-b4b72b051df3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:16:22.290346 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da00536-1428-4cd3-b08c-b4b72b051df3 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:16:22.299646 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 791/1586] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Aug 24 06:16:22 2022] POST /identity/v3/users => generated 294 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:16:22.301037 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128896c1-1de3-4339-881d-30cffb273c64 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:16:22.304381 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=90,keystone:INSERT=10 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:16:22.310920 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128896c1-1de3-4339-881d-30cffb273c64 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:16:22.315619 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca72d4f-b89a-4e98-aa06-c26fd6e3a2c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:16:22.316144 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca72d4f-b89a-4e98-aa06-c26fd6e3a2c9 admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:22.316426 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca72d4f-b89a-4e98-aa06-c26fd6e3a2c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:22.317248 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca72d4f-b89a-4e98-aa06-c26fd6e3a2c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:22.317876 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca72d4f-b89a-4e98-aa06-c26fd6e3a2c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:16:22.319528 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca72d4f-b89a-4e98-aa06-c26fd6e3a2c9 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:16:22.329934 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 792/1587] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Aug 24 06:16:22 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:16:22.330742 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca72d4f-b89a-4e98-aa06-c26fd6e3a2c9 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:16:22.340778 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca72d4f-b89a-4e98-aa06-c26fd6e3a2c9 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:16:22.345361 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc6e717-132f-4e6a-bd6a-824210a897d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:16:22.345981 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc6e717-132f-4e6a-bd6a-824210a897d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:22.346262 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc6e717-132f-4e6a-bd6a-824210a897d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:22.346538 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc6e717-132f-4e6a-bd6a-824210a897d6 admin admin] PATH_INFO: `/v3/projects/2527202c380e4491b7509f3b8a6fd61d/users/75db228aee9b4903aadebb69713cc36d/roles/580aa7dbb38d4cff8f9d1bb8983f99ef` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:22.360636 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc6e717-132f-4e6a-bd6a-824210a897d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2527202c380e4491b7509f3b8a6fd61d, user_id=75db228aee9b4903aadebb69713cc36d, role_id=580aa7dbb38d4cff8f9d1bb8983f99ef)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:16:22.363270 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc6e717-132f-4e6a-bd6a-824210a897d6 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:16:22.409137 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 796/1588] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Aug 24 06:16:22 2022] POST /identity/v3/users => generated 291 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:16:22.410261 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da00536-1428-4cd3-b08c-b4b72b051df3 admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:16:22.419857 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da00536-1428-4cd3-b08c-b4b72b051df3 admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:16:22.424659 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e480a712-db22-4aa1-b079-0e2a0ea02174 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:16:22.425513 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e480a712-db22-4aa1-b079-0e2a0ea02174 admin admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:22.426315 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e480a712-db22-4aa1-b079-0e2a0ea02174 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:22.426589 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e480a712-db22-4aa1-b079-0e2a0ea02174 admin admin] PATH_INFO: `/v3/roles` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:22.427230 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e480a712-db22-4aa1-b079-0e2a0ea02174 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:16:22.428868 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e480a712-db22-4aa1-b079-0e2a0ea02174 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:16:22.440123 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 797/1589] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Aug 24 06:16:22 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:16:22.441101 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e480a712-db22-4aa1-b079-0e2a0ea02174 admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:16:22.453060 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e480a712-db22-4aa1-b079-0e2a0ea02174 admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:16:22.457847 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b64eba-e722-48c7-bb89-645bbc717d53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:16:22.458446 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b64eba-e722-48c7-bb89-645bbc717d53 admin admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:22.458731 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b64eba-e722-48c7-bb89-645bbc717d53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:22.459014 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b64eba-e722-48c7-bb89-645bbc717d53 admin admin] PATH_INFO: `/v3/roles` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:22.459645 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b64eba-e722-48c7-bb89-645bbc717d53 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:16:22.461288 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b64eba-e722-48c7-bb89-645bbc717d53 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:16:22.466514 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 793/1590] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Aug 24 06:16:22 2022] PUT /identity/v3/projects/2527202c380e4491b7509f3b8a6fd61d/users/75db228aee9b4903aadebb69713cc36d/roles/580aa7dbb38d4cff8f9d1bb8983f99ef => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:16:22.469772 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc6e717-132f-4e6a-bd6a-824210a897d6 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:16:22.472174 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 798/1591] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Aug 24 06:16:22 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:16:22.475924 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b64eba-e722-48c7-bb89-645bbc717d53 admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:16:22.485675 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b64eba-e722-48c7-bb89-645bbc717d53 admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:16:22.560788 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7252d128-e370-4c82-864c-d6980f8b94e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:16:22.561828 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7252d128-e370-4c82-864c-d6980f8b94e7 admin admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:22.564831 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7252d128-e370-4c82-864c-d6980f8b94e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:22.565456 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7252d128-e370-4c82-864c-d6980f8b94e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:22.566560 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7252d128-e370-4c82-864c-d6980f8b94e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:16:22.568808 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7252d128-e370-4c82-864c-d6980f8b94e7 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:16:22.571101 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc6e717-132f-4e6a-bd6a-824210a897d6 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:16:22.576113 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b825dc34-86df-4a6e-83d9-25baca7562c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:16:22.576945 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b825dc34-86df-4a6e-83d9-25baca7562c2 admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:22.577562 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b825dc34-86df-4a6e-83d9-25baca7562c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:22.577845 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b825dc34-86df-4a6e-83d9-25baca7562c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:22.578499 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b825dc34-86df-4a6e-83d9-25baca7562c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:16:22.580226 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b825dc34-86df-4a6e-83d9-25baca7562c2 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:16:22.580472 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 799/1592] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Aug 24 06:16:22 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:16:22.581395 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7252d128-e370-4c82-864c-d6980f8b94e7 admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:16:22.593742 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 794/1593] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Aug 24 06:16:22 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:16:22.594066 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7252d128-e370-4c82-864c-d6980f8b94e7 admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:16:22.595123 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b825dc34-86df-4a6e-83d9-25baca7562c2 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:16:22.602111 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f94177-f16c-463c-b8c9-87f919769ac6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:16:22.602783 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f94177-f16c-463c-b8c9-87f919769ac6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:22.603066 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f94177-f16c-463c-b8c9-87f919769ac6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:22.603346 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f94177-f16c-463c-b8c9-87f919769ac6 admin admin] PATH_INFO: `/v3/projects/ed8aa953e25943a197bec8ac5148a39c/users/6021a225b5fd4377a9f8120a84232709/roles/580aa7dbb38d4cff8f9d1bb8983f99ef` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:22.605528 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b825dc34-86df-4a6e-83d9-25baca7562c2 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:16:22.613898 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42989932-05be-466c-8c3d-13a43dcc2490 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:16:22.614115 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42989932-05be-466c-8c3d-13a43dcc2490 admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:22.614201 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42989932-05be-466c-8c3d-13a43dcc2490 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:22.614779 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42989932-05be-466c-8c3d-13a43dcc2490 admin admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:22.615435 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42989932-05be-466c-8c3d-13a43dcc2490 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:16:22.617839 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42989932-05be-466c-8c3d-13a43dcc2490 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:16:22.619447 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f94177-f16c-463c-b8c9-87f919769ac6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed8aa953e25943a197bec8ac5148a39c, user_id=6021a225b5fd4377a9f8120a84232709, role_id=580aa7dbb38d4cff8f9d1bb8983f99ef)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:16:22.625757 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f94177-f16c-463c-b8c9-87f919769ac6 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:16:22.636064 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 795/1594] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Aug 24 06:16:22 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:16:22.636064 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42989932-05be-466c-8c3d-13a43dcc2490 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:16:22.644805 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42989932-05be-466c-8c3d-13a43dcc2490 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:16:22.650526 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c65853-4e0b-4fb9-968c-b77de7041402 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:16:22.654486 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c65853-4e0b-4fb9-968c-b77de7041402 admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:22.654486 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c65853-4e0b-4fb9-968c-b77de7041402 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:22.654486 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c65853-4e0b-4fb9-968c-b77de7041402 admin admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:22.654486 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c65853-4e0b-4fb9-968c-b77de7041402 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:16:22.654486 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c65853-4e0b-4fb9-968c-b77de7041402 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:16:22.666716 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 796/1595] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Aug 24 06:16:22 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:16:22.667702 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d1efca-1910-476c-a359-96e9d950e343 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:22.667984 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d1efca-1910-476c-a359-96e9d950e343 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:22.668258 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d1efca-1910-476c-a359-96e9d950e343 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:22.690840 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:16:22.694724 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-82d1efca-1910-476c-a359-96e9d950e343 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:16:22.696023 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-82d1efca-1910-476c-a359-96e9d950e343 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:16:22.714797 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 797/1596] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:16:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:16:22.714797 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d1efca-1910-476c-a359-96e9d950e343 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:16:22.731676 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d1efca-1910-476c-a359-96e9d950e343 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:16:22.737132 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b246af-e32d-4003-ac9f-d4c7b3d4cc32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:16:22.737800 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b246af-e32d-4003-ac9f-d4c7b3d4cc32 admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:22.738077 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b246af-e32d-4003-ac9f-d4c7b3d4cc32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:22.738370 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b246af-e32d-4003-ac9f-d4c7b3d4cc32 admin admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:22.739768 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b246af-e32d-4003-ac9f-d4c7b3d4cc32 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:16:22.741723 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b246af-e32d-4003-ac9f-d4c7b3d4cc32 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:16:22.751295 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 800/1597] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Aug 24 06:16:22 2022] PUT /identity/v3/projects/ed8aa953e25943a197bec8ac5148a39c/users/6021a225b5fd4377a9f8120a84232709/roles/580aa7dbb38d4cff8f9d1bb8983f99ef => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:16:22.752660 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f94177-f16c-463c-b8c9-87f919769ac6 admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:16:22.757415 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 798/1598] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Aug 24 06:16:22 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:16:22.757415 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b246af-e32d-4003-ac9f-d4c7b3d4cc32 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:16:22.767100 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b246af-e32d-4003-ac9f-d4c7b3d4cc32 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:16:22.827993 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f94177-f16c-463c-b8c9-87f919769ac6 admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:16:22.833591 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd3bdc7-c7d1-431a-aa89-03879535e0b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:16:22.839543 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544d79b1-982a-48f7-8d9f-68f3f9634f35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:16:22.839836 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd3bdc7-c7d1-431a-aa89-03879535e0b5 admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:22.839836 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd3bdc7-c7d1-431a-aa89-03879535e0b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:22.840033 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544d79b1-982a-48f7-8d9f-68f3f9634f35 admin admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:22.840123 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd3bdc7-c7d1-431a-aa89-03879535e0b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:22.840216 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544d79b1-982a-48f7-8d9f-68f3f9634f35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:22.840216 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544d79b1-982a-48f7-8d9f-68f3f9634f35 admin admin] PATH_INFO: `/v3/roles` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:22.840384 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd3bdc7-c7d1-431a-aa89-03879535e0b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:16:22.840468 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544d79b1-982a-48f7-8d9f-68f3f9634f35 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:16:22.840575 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd3bdc7-c7d1-431a-aa89-03879535e0b5 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:16:22.840673 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544d79b1-982a-48f7-8d9f-68f3f9634f35 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:16:22.856015 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 799/1599] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Aug 24 06:16:22 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:16:22.856015 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd3bdc7-c7d1-431a-aa89-03879535e0b5 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:16:22.856310 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 801/1600] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Aug 24 06:16:22 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:16:22.856310 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544d79b1-982a-48f7-8d9f-68f3f9634f35 admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:16:22.862184 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd3bdc7-c7d1-431a-aa89-03879535e0b5 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:16:22.867403 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544d79b1-982a-48f7-8d9f-68f3f9634f35 admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:16:22.869713 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad406c7-4d6c-4899-b0cc-1a9326cc9464 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:16:22.870439 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad406c7-4d6c-4899-b0cc-1a9326cc9464 admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:22.870640 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd57f1fd-2b07-4884-92f3-4fe35273573e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:16:22.870760 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad406c7-4d6c-4899-b0cc-1a9326cc9464 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:22.871051 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad406c7-4d6c-4899-b0cc-1a9326cc9464 admin admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:22.871596 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd57f1fd-2b07-4884-92f3-4fe35273573e admin admin] REQUEST_METHOD: `PUT` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:22.871719 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad406c7-4d6c-4899-b0cc-1a9326cc9464 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:16:22.872055 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd57f1fd-2b07-4884-92f3-4fe35273573e admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:22.872425 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd57f1fd-2b07-4884-92f3-4fe35273573e admin admin] PATH_INFO: `/v3/projects/98863eced91a48beba0ea96d23fb851d/users/43ca918762024b82b73a54393a384533/roles/580aa7dbb38d4cff8f9d1bb8983f99ef` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:22.873678 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad406c7-4d6c-4899-b0cc-1a9326cc9464 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:16:22.885745 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 800/1601] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Aug 24 06:16:22 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:16:22.889783 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad406c7-4d6c-4899-b0cc-1a9326cc9464 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:16:22.890145 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd57f1fd-2b07-4884-92f3-4fe35273573e admin admin] RBAC: Authorizing `identity:create_grant(project_id=98863eced91a48beba0ea96d23fb851d, user_id=43ca918762024b82b73a54393a384533, role_id=580aa7dbb38d4cff8f9d1bb8983f99ef)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:16:22.892982 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd57f1fd-2b07-4884-92f3-4fe35273573e admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:16:22.949532 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad406c7-4d6c-4899-b0cc-1a9326cc9464 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:16:22.953274 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86476cc-14c3-47af-90ed-e5f382903039 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bb6c79db75e4d169331c02133d1ec32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:16:22.953880 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86476cc-14c3-47af-90ed-e5f382903039 service neutron] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:22.957386 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86476cc-14c3-47af-90ed-e5f382903039 service neutron] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:22.957386 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86476cc-14c3-47af-90ed-e5f382903039 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:22.964362 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86476cc-14c3-47af-90ed-e5f382903039 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:16:22.965898 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86476cc-14c3-47af-90ed-e5f382903039 service neutron] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:16:22.978350 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 801/1602] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:16:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:16:22.979651 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6532de1-12d2-42a3-af73-9d0cccce6a92 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:22.979937 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6532de1-12d2-42a3-af73-9d0cccce6a92 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:22.980238 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6532de1-12d2-42a3-af73-9d0cccce6a92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:22.999790 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-a6532de1-12d2-42a3-af73-9d0cccce6a92 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:16:23.001477 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-a6532de1-12d2-42a3-af73-9d0cccce6a92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:16:23.011719 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 802/1603] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:16:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:16:23.012407 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6532de1-12d2-42a3-af73-9d0cccce6a92 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:16:23.023754 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 802/1604] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Aug 24 06:16:22 2022] PUT /identity/v3/projects/98863eced91a48beba0ea96d23fb851d/users/43ca918762024b82b73a54393a384533/roles/580aa7dbb38d4cff8f9d1bb8983f99ef => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:16:23.031287 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6532de1-12d2-42a3-af73-9d0cccce6a92 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:16:23.032563 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd57f1fd-2b07-4884-92f3-4fe35273573e admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:16:23.066613 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:16:23.114588 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e51a34f-4438-4bd1-be27-3cdab3c2d8be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:16:23.115367 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e51a34f-4438-4bd1-be27-3cdab3c2d8be admin admin] REQUEST_METHOD: `PUT` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:23.115751 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e51a34f-4438-4bd1-be27-3cdab3c2d8be admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:23.116146 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e51a34f-4438-4bd1-be27-3cdab3c2d8be admin admin] PATH_INFO: `/v3/projects/038bd6c34b4b4881a5b9517b458b2c50/users/dbcbc15f5edb41d4bf58299ea45a62bd/roles/580aa7dbb38d4cff8f9d1bb8983f99ef` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:23.128637 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd57f1fd-2b07-4884-92f3-4fe35273573e admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:16:23.134600 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe69c65-d6b5-4df1-9c14-70373bb3a1dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:16:23.134886 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e51a34f-4438-4bd1-be27-3cdab3c2d8be admin admin] RBAC: Authorizing `identity:create_grant(project_id=038bd6c34b4b4881a5b9517b458b2c50, user_id=dbcbc15f5edb41d4bf58299ea45a62bd, role_id=580aa7dbb38d4cff8f9d1bb8983f99ef)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:16:23.135574 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe69c65-d6b5-4df1-9c14-70373bb3a1dc admin admin] REQUEST_METHOD: `PUT` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:23.135574 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe69c65-d6b5-4df1-9c14-70373bb3a1dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:23.135873 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe69c65-d6b5-4df1-9c14-70373bb3a1dc admin admin] PATH_INFO: `/v3/projects/b1f2c78322cb4c999f1e01b82b81a884/users/6abfd0a28f9e482badae7018f1b7f9a1/roles/580aa7dbb38d4cff8f9d1bb8983f99ef` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:23.137664 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e51a34f-4438-4bd1-be27-3cdab3c2d8be admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:16:23.153222 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe69c65-d6b5-4df1-9c14-70373bb3a1dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=b1f2c78322cb4c999f1e01b82b81a884, user_id=6abfd0a28f9e482badae7018f1b7f9a1, role_id=580aa7dbb38d4cff8f9d1bb8983f99ef)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:16:23.156203 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe69c65-d6b5-4df1-9c14-70373bb3a1dc admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:16:23.249355 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 803/1605] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Aug 24 06:16:23 2022] PUT /identity/v3/projects/038bd6c34b4b4881a5b9517b458b2c50/users/dbcbc15f5edb41d4bf58299ea45a62bd/roles/580aa7dbb38d4cff8f9d1bb8983f99ef => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:16:23.250712 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e51a34f-4438-4bd1-be27-3cdab3c2d8be admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:16:23.301614 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:16:23.316793 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 803/1606] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Aug 24 06:16:23 2022] PUT /identity/v3/projects/b1f2c78322cb4c999f1e01b82b81a884/users/6abfd0a28f9e482badae7018f1b7f9a1/roles/580aa7dbb38d4cff8f9d1bb8983f99ef => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:16:23.316793 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe69c65-d6b5-4df1-9c14-70373bb3a1dc admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:16:23.348558 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e51a34f-4438-4bd1-be27-3cdab3c2d8be admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:16:23.407133 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe69c65-d6b5-4df1-9c14-70373bb3a1dc admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:16:23.414693 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd601bf-7a24-44b4-95e3-104984d42c03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:16:23.415498 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd601bf-7a24-44b4-95e3-104984d42c03 admin admin] REQUEST_METHOD: `PUT` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:23.415715 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd601bf-7a24-44b4-95e3-104984d42c03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:23.415983 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd601bf-7a24-44b4-95e3-104984d42c03 admin admin] PATH_INFO: `/v3/projects/00d93c0b6f064311b732714aa91a96bb/users/114a88fb0dfe4f33abf5a654dee0df1e/roles/580aa7dbb38d4cff8f9d1bb8983f99ef` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:23.434051 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd601bf-7a24-44b4-95e3-104984d42c03 admin admin] RBAC: Authorizing `identity:create_grant(project_id=00d93c0b6f064311b732714aa91a96bb, user_id=114a88fb0dfe4f33abf5a654dee0df1e, role_id=580aa7dbb38d4cff8f9d1bb8983f99ef)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:16:23.436858 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd601bf-7a24-44b4-95e3-104984d42c03 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:16:23.452220 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d60eb9-2e9b-4c20-a1ab-25ff0008607f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:16:23.453600 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d60eb9-2e9b-4c20-a1ab-25ff0008607f admin admin] REQUEST_METHOD: `PUT` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:23.454447 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d60eb9-2e9b-4c20-a1ab-25ff0008607f admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:23.454906 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d60eb9-2e9b-4c20-a1ab-25ff0008607f admin admin] PATH_INFO: `/v3/projects/f4e8d9878aff47cea1bd895f618badb8/users/5059723f417445fa934ecfe3e50d0197/roles/580aa7dbb38d4cff8f9d1bb8983f99ef` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:23.477277 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d60eb9-2e9b-4c20-a1ab-25ff0008607f admin admin] RBAC: Authorizing `identity:create_grant(project_id=f4e8d9878aff47cea1bd895f618badb8, user_id=5059723f417445fa934ecfe3e50d0197, role_id=580aa7dbb38d4cff8f9d1bb8983f99ef)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:16:23.480151 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d60eb9-2e9b-4c20-a1ab-25ff0008607f admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:16:23.506609 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 804/1607] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Aug 24 06:16:23 2022] PUT /identity/v3/projects/00d93c0b6f064311b732714aa91a96bb/users/114a88fb0dfe4f33abf5a654dee0df1e/roles/580aa7dbb38d4cff8f9d1bb8983f99ef => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:16:23.507160 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd601bf-7a24-44b4-95e3-104984d42c03 admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:16:23.510547 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-7cd601bf-7a24-44b4-95e3-104984d42c03 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:16:23.597113 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 804/1608] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Aug 24 06:16:23 2022] PUT /identity/v3/projects/f4e8d9878aff47cea1bd895f618badb8/users/5059723f417445fa934ecfe3e50d0197/roles/580aa7dbb38d4cff8f9d1bb8983f99ef => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:16:23.597113 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2db1ba-da0c-447d-a5b3-aa8b97caa9f7 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:23.597113 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2db1ba-da0c-447d-a5b3-aa8b97caa9f7 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:23.597113 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2db1ba-da0c-447d-a5b3-aa8b97caa9f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:23.634086 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd601bf-7a24-44b4-95e3-104984d42c03 admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:16:23.656966 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-5a2db1ba-da0c-447d-a5b3-aa8b97caa9f7 None None] MFA Rules not processed for user `43ca918762024b82b73a54393a384533`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:16:23.658258 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-5a2db1ba-da0c-447d-a5b3-aa8b97caa9f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:16:23.716597 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:16:23.750091 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80052d9-6c50-45be-b58b-dccf4c57e2a4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bb6c79db75e4d169331c02133d1ec32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:16:23.750768 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80052d9-6c50-45be-b58b-dccf4c57e2a4 service neutron] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:23.751050 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80052d9-6c50-45be-b58b-dccf4c57e2a4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:23.763781 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80052d9-6c50-45be-b58b-dccf4c57e2a4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:23.771553 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 805/1609] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:16:23 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:16:23.772860 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58c813b-4212-4310-9cef-58c065a91d0f None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:23.773359 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58c813b-4212-4310-9cef-58c065a91d0f None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:23.774171 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58c813b-4212-4310-9cef-58c065a91d0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:23.807738 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:16:23.812797 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-d58c813b-4212-4310-9cef-58c065a91d0f None None] MFA Rules not processed for user `dbcbc15f5edb41d4bf58299ea45a62bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:16:23.814296 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-d58c813b-4212-4310-9cef-58c065a91d0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:16:23.869338 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80052d9-6c50-45be-b58b-dccf4c57e2a4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:16:23.871031 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80052d9-6c50-45be-b58b-dccf4c57e2a4 service neutron] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:16:23.883652 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 805/1610] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:16:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:16:23.885349 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c735024-ae69-45a5-805e-7fb6680d7860 None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:23.885864 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c735024-ae69-45a5-805e-7fb6680d7860 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:23.886253 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c735024-ae69-45a5-805e-7fb6680d7860 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:23.915738 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-9c735024-ae69-45a5-805e-7fb6680d7860 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:16:23.923192 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-9c735024-ae69-45a5-805e-7fb6680d7860 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:16:23.946883 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 806/1611] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:16:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:16:23.950412 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dcc45d-72e3-490e-97ad-bd37550ba6f1 None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:23.950706 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dcc45d-72e3-490e-97ad-bd37550ba6f1 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:23.950975 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dcc45d-72e3-490e-97ad-bd37550ba6f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:23.969320 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 806/1612] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:16:23 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:16:23.971106 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a968e04-84ee-4e1a-9383-37c16ca7e875 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:23.971106 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a968e04-84ee-4e1a-9383-37c16ca7e875 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:23.971106 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a968e04-84ee-4e1a-9383-37c16ca7e875 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:24.011605 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-5a968e04-84ee-4e1a-9383-37c16ca7e875 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:16:24.015135 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-5a968e04-84ee-4e1a-9383-37c16ca7e875 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:16:24.020299 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-e4dcc45d-72e3-490e-97ad-bd37550ba6f1 None None] MFA Rules not processed for user `114a88fb0dfe4f33abf5a654dee0df1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:16:24.022169 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-e4dcc45d-72e3-490e-97ad-bd37550ba6f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:16:24.033208 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 807/1613] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:16:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:16:24.034522 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f14e07-de8d-4eff-860e-e1598114fbd7 None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:24.034809 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f14e07-de8d-4eff-860e-e1598114fbd7 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:24.035198 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f14e07-de8d-4eff-860e-e1598114fbd7 None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:24.041386 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 808/1614] 10.222.0.62 () {54 vars in 962 bytes} [Wed Aug 24 06:16:24 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:16:24.042249 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f14e07-de8d-4eff-860e-e1598114fbd7 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:16:24.077159 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f14e07-de8d-4eff-860e-e1598114fbd7 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:16:24.086079 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a57dae-0a9d-4efd-85bc-741260ff1be8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:16:24.087520 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a57dae-0a9d-4efd-85bc-741260ff1be8 admin admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:24.088161 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a57dae-0a9d-4efd-85bc-741260ff1be8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:24.088447 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a57dae-0a9d-4efd-85bc-741260ff1be8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:24.089332 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a57dae-0a9d-4efd-85bc-741260ff1be8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:16:24.092343 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a57dae-0a9d-4efd-85bc-741260ff1be8 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:16:24.130697 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 807/1615] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:16:23 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:16:24.131685 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4dcc45d-72e3-490e-97ad-bd37550ba6f1 None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:16:24.142340 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4dcc45d-72e3-490e-97ad-bd37550ba6f1 None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:16:24.151035 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66ef037-c7f4-4a13-ae16-19d968668f83 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bb6c79db75e4d169331c02133d1ec32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:16:24.151035 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66ef037-c7f4-4a13-ae16-19d968668f83 service neutron] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:24.151035 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66ef037-c7f4-4a13-ae16-19d968668f83 service neutron] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:24.151035 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66ef037-c7f4-4a13-ae16-19d968668f83 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:24.160279 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66ef037-c7f4-4a13-ae16-19d968668f83 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:16:24.162030 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66ef037-c7f4-4a13-ae16-19d968668f83 service neutron] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:16:24.172070 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 808/1616] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:16:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:16:24.178397 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea5d9c7-d3a3-4245-b61a-50aecdcd82e7 None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:24.178727 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea5d9c7-d3a3-4245-b61a-50aecdcd82e7 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:24.179370 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea5d9c7-d3a3-4245-b61a-50aecdcd82e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:24.223796 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 809/1617] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Aug 24 06:16:24 2022] POST /identity/v3/projects => generated 401 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:16:24.227071 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a57dae-0a9d-4efd-85bc-741260ff1be8 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:16:24.239467 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-4ea5d9c7-d3a3-4245-b61a-50aecdcd82e7 None None] MFA Rules not processed for user `fb6d3743340241f98cacb9e0d063298c`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:16:24.241067 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-4ea5d9c7-d3a3-4245-b61a-50aecdcd82e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:16:24.316417 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:16:24.375655 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a57dae-0a9d-4efd-85bc-741260ff1be8 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:16:24.378570 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 809/1618] 10.222.0.62 () {58 vars in 1059 bytes} [Wed Aug 24 06:16:24 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:16:24.379800 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea5d9c7-d3a3-4245-b61a-50aecdcd82e7 None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:16:24.384400 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7579dd-ded9-4b32-88a5-964d2d88df9d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bb6c79db75e4d169331c02133d1ec32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:16:24.386175 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7579dd-ded9-4b32-88a5-964d2d88df9d service neutron] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:24.386465 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7579dd-ded9-4b32-88a5-964d2d88df9d service neutron] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:24.389004 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7579dd-ded9-4b32-88a5-964d2d88df9d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:24.489478 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7579dd-ded9-4b32-88a5-964d2d88df9d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:16:24.493944 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7579dd-ded9-4b32-88a5-964d2d88df9d service neutron] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:16:24.496305 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea5d9c7-d3a3-4245-b61a-50aecdcd82e7 None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:16:24.502776 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1210521e-9e7c-4169-b23e-9fe0018cf9a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:16:24.503449 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1210521e-9e7c-4169-b23e-9fe0018cf9a6 admin admin] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:24.504035 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1210521e-9e7c-4169-b23e-9fe0018cf9a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:24.504307 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1210521e-9e7c-4169-b23e-9fe0018cf9a6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:24.505159 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1210521e-9e7c-4169-b23e-9fe0018cf9a6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:16:24.507292 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1210521e-9e7c-4169-b23e-9fe0018cf9a6 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:16:24.508387 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 810/1619] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:16:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 283 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:16:24.509563 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7579dd-ded9-4b32-88a5-964d2d88df9d service neutron] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:16:24.524995 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7579dd-ded9-4b32-88a5-964d2d88df9d service neutron] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:16:24.529629 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2ec37a-1fdc-43cc-b31a-189b9e2d5227 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:16:24.530262 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2ec37a-1fdc-43cc-b31a-189b9e2d5227 admin admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:24.530542 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2ec37a-1fdc-43cc-b31a-189b9e2d5227 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:24.530981 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2ec37a-1fdc-43cc-b31a-189b9e2d5227 admin admin] PATH_INFO: `/v3/users` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:24.532220 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2ec37a-1fdc-43cc-b31a-189b9e2d5227 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:16:24.533678 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2ec37a-1fdc-43cc-b31a-189b9e2d5227 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:16:24.633619 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 811/1620] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Aug 24 06:16:24 2022] POST /identity/v3/users => generated 307 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:16:24.634774 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2ec37a-1fdc-43cc-b31a-189b9e2d5227 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:16:24.641519 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 810/1621] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Aug 24 06:16:24 2022] POST /identity/v3/projects => generated 381 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:16:24.648159 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1210521e-9e7c-4169-b23e-9fe0018cf9a6 admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:16:24.666429 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2ec37a-1fdc-43cc-b31a-189b9e2d5227 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:16:24.848640 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1210521e-9e7c-4169-b23e-9fe0018cf9a6 admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:16:24.918436 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888c7760-78f7-4dc8-a93f-50e1bd4e16dc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb6d3743340241f98cacb9e0d063298c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:16:24.918436 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888c7760-78f7-4dc8-a93f-50e1bd4e16dc service nova] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:24.918436 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888c7760-78f7-4dc8-a93f-50e1bd4e16dc service nova] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:24.918436 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888c7760-78f7-4dc8-a93f-50e1bd4e16dc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:24.918969 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246378b7-ef65-4829-9082-1cf1547be633 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:16:24.918969 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246378b7-ef65-4829-9082-1cf1547be633 admin admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:24.918969 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246378b7-ef65-4829-9082-1cf1547be633 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:24.918969 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246378b7-ef65-4829-9082-1cf1547be633 admin admin] PATH_INFO: `/v3/roles` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:24.919775 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246378b7-ef65-4829-9082-1cf1547be633 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:16:24.919775 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246378b7-ef65-4829-9082-1cf1547be633 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:16:24.919775 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 811/1622] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Aug 24 06:16:24 2022] GET /identity/v3/roles => generated 1420 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:16:24.919775 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246378b7-ef65-4829-9082-1cf1547be633 admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:16:24.919775 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246378b7-ef65-4829-9082-1cf1547be633 admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:16:24.920328 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbb4aa5-3f4c-4d36-85b4-23fbd537418e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:16:24.920328 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbb4aa5-3f4c-4d36-85b4-23fbd537418e admin admin] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:24.920328 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbb4aa5-3f4c-4d36-85b4-23fbd537418e admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:24.920328 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbb4aa5-3f4c-4d36-85b4-23fbd537418e admin admin] PATH_INFO: `/v3/users` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:24.920993 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbb4aa5-3f4c-4d36-85b4-23fbd537418e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:16:24.920993 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbb4aa5-3f4c-4d36-85b4-23fbd537418e admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:16:24.921223 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:16:24.921223 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888c7760-78f7-4dc8-a93f-50e1bd4e16dc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:16:24.921223 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888c7760-78f7-4dc8-a93f-50e1bd4e16dc service nova] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:16:24.921223 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 812/1623] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:16:24 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 266 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:16:24.921223 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888c7760-78f7-4dc8-a93f-50e1bd4e16dc service nova] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:16:24.921223 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888c7760-78f7-4dc8-a93f-50e1bd4e16dc service nova] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:16:24.935908 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b0195b-a1be-404d-ab4f-a0b544ceb22e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:16:24.936681 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b0195b-a1be-404d-ab4f-a0b544ceb22e admin admin] REQUEST_METHOD: `PUT` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:24.936971 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b0195b-a1be-404d-ab4f-a0b544ceb22e admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:24.937793 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b0195b-a1be-404d-ab4f-a0b544ceb22e admin admin] PATH_INFO: `/v3/projects/ce6c24088632410ba262a9e202c61fa8/users/5d75f562443b4cd2a88aa37e268c8851/roles/49e47cd157d743f6905f1db07ec878a7` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:24.981012 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b0195b-a1be-404d-ab4f-a0b544ceb22e admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce6c24088632410ba262a9e202c61fa8, user_id=5d75f562443b4cd2a88aa37e268c8851, role_id=49e47cd157d743f6905f1db07ec878a7)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:16:24.987652 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 812/1624] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Aug 24 06:16:24 2022] POST /identity/v3/users => generated 297 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:16:24.991038 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648384ad-a59d-4358-8e4f-d70a71f23633 None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:24.991038 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648384ad-a59d-4358-8e4f-d70a71f23633 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:24.991038 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648384ad-a59d-4358-8e4f-d70a71f23633 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:24.993679 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b0195b-a1be-404d-ab4f-a0b544ceb22e admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:16:25.068925 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-648384ad-a59d-4358-8e4f-d70a71f23633 None None] MFA Rules not processed for user `75db228aee9b4903aadebb69713cc36d`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:16:25.072581 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-648384ad-a59d-4358-8e4f-d70a71f23633 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:16:25.113291 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 813/1625] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Aug 24 06:16:24 2022] PUT /identity/v3/projects/ce6c24088632410ba262a9e202c61fa8/users/5d75f562443b4cd2a88aa37e268c8851/roles/49e47cd157d743f6905f1db07ec878a7 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:16:25.114138 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b0195b-a1be-404d-ab4f-a0b544ceb22e admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:16:25.152266 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:16:25.215300 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 813/1626] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:16:24 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:16:25.217464 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648384ad-a59d-4358-8e4f-d70a71f23633 None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:16:25.233227 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648384ad-a59d-4358-8e4f-d70a71f23633 None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:16:25.254398 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b0195b-a1be-404d-ab4f-a0b544ceb22e admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:16:25.267323 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f228faea-dc7d-4cf8-bfcd-94de6302f7a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:16:25.267947 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f228faea-dc7d-4cf8-bfcd-94de6302f7a1 admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:25.268229 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f228faea-dc7d-4cf8-bfcd-94de6302f7a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:25.268802 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f228faea-dc7d-4cf8-bfcd-94de6302f7a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:25.269170 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f228faea-dc7d-4cf8-bfcd-94de6302f7a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:16:25.271904 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f228faea-dc7d-4cf8-bfcd-94de6302f7a1 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:16:25.294736 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 814/1627] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Aug 24 06:16:25 2022] GET /identity/v3/roles => generated 1420 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:16:25.295588 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f228faea-dc7d-4cf8-bfcd-94de6302f7a1 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:16:25.315854 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f228faea-dc7d-4cf8-bfcd-94de6302f7a1 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:16:25.335188 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a293409d-c71a-4529-87e4-dea480bdccc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:16:25.335188 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a293409d-c71a-4529-87e4-dea480bdccc1 admin admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:25.340758 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a293409d-c71a-4529-87e4-dea480bdccc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:25.340758 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a293409d-c71a-4529-87e4-dea480bdccc1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:25.340758 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a293409d-c71a-4529-87e4-dea480bdccc1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:16:25.340758 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a293409d-c71a-4529-87e4-dea480bdccc1 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:16:25.358228 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d286b0-f8a4-47bc-80ac-f1737a2b884a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:16:25.358946 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d286b0-f8a4-47bc-80ac-f1737a2b884a admin admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:25.359294 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d286b0-f8a4-47bc-80ac-f1737a2b884a admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:25.359646 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d286b0-f8a4-47bc-80ac-f1737a2b884a admin admin] PATH_INFO: `/v3/roles` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:25.360372 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d286b0-f8a4-47bc-80ac-f1737a2b884a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:16:25.364020 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d286b0-f8a4-47bc-80ac-f1737a2b884a admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:16:25.389748 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 814/1628] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Aug 24 06:16:25 2022] GET /identity/v3/roles => generated 1420 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:16:25.391335 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d286b0-f8a4-47bc-80ac-f1737a2b884a admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:16:25.407029 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d286b0-f8a4-47bc-80ac-f1737a2b884a admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:16:25.415590 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7e52f3-2561-44bc-a32a-63343094cd15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:16:25.421183 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7e52f3-2561-44bc-a32a-63343094cd15 admin admin] REQUEST_METHOD: `PUT` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:25.421183 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7e52f3-2561-44bc-a32a-63343094cd15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:25.421183 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7e52f3-2561-44bc-a32a-63343094cd15 admin admin] PATH_INFO: `/v3/projects/b48751354827457e8ba7f0e71362f0de/users/16d019e12c164a208003ba5710c7ea15/roles/49e47cd157d743f6905f1db07ec878a7` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:25.456093 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7e52f3-2561-44bc-a32a-63343094cd15 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b48751354827457e8ba7f0e71362f0de, user_id=16d019e12c164a208003ba5710c7ea15, role_id=49e47cd157d743f6905f1db07ec878a7)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:16:25.462314 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7e52f3-2561-44bc-a32a-63343094cd15 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:16:25.491460 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 815/1629] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Aug 24 06:16:25 2022] POST /identity/v3/projects => generated 403 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:16:25.492300 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a293409d-c71a-4529-87e4-dea480bdccc1 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:16:25.567159 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 815/1630] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Aug 24 06:16:25 2022] PUT /identity/v3/projects/b48751354827457e8ba7f0e71362f0de/users/16d019e12c164a208003ba5710c7ea15/roles/49e47cd157d743f6905f1db07ec878a7 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:16:25.567159 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7e52f3-2561-44bc-a32a-63343094cd15 admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:16:25.625509 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a293409d-c71a-4529-87e4-dea480bdccc1 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:16:25.736050 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7e52f3-2561-44bc-a32a-63343094cd15 admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:16:25.756113 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e416e0-d060-4285-9a06-725c3eda239e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:16:25.756778 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e416e0-d060-4285-9a06-725c3eda239e admin admin] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:25.757036 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e416e0-d060-4285-9a06-725c3eda239e admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:25.757320 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e416e0-d060-4285-9a06-725c3eda239e admin admin] PATH_INFO: `/v3/users` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:25.757947 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e416e0-d060-4285-9a06-725c3eda239e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:16:25.759957 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e416e0-d060-4285-9a06-725c3eda239e admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:16:26.030924 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3e920c-ea9c-4eb9-9e82-708173b8a740 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:16:26.030924 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3e920c-ea9c-4eb9-9e82-708173b8a740 admin admin] REQUEST_METHOD: `PUT` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:26.030924 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3e920c-ea9c-4eb9-9e82-708173b8a740 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:26.030924 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3e920c-ea9c-4eb9-9e82-708173b8a740 admin admin] PATH_INFO: `/v3/projects/ce6c24088632410ba262a9e202c61fa8/users/5d75f562443b4cd2a88aa37e268c8851/roles/49e47cd157d743f6905f1db07ec878a7` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:26.032983 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3e920c-ea9c-4eb9-9e82-708173b8a740 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce6c24088632410ba262a9e202c61fa8, user_id=5d75f562443b4cd2a88aa37e268c8851, role_id=49e47cd157d743f6905f1db07ec878a7)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:16:26.032983 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3e920c-ea9c-4eb9-9e82-708173b8a740 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:16:26.032983 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 816/1631] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Aug 24 06:16:25 2022] PUT /identity/v3/projects/ce6c24088632410ba262a9e202c61fa8/users/5d75f562443b4cd2a88aa37e268c8851/roles/49e47cd157d743f6905f1db07ec878a7 => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:16:26.032983 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3e920c-ea9c-4eb9-9e82-708173b8a740 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:16:26.059423 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 816/1632] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Aug 24 06:16:25 2022] POST /identity/v3/users => generated 308 bytes in 325 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:16:26.059423 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5691d4d-ffa2-48d2-9a75-8acbe857e2e5 None None] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:26.059423 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5691d4d-ffa2-48d2-9a75-8acbe857e2e5 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:26.059423 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5691d4d-ffa2-48d2-9a75-8acbe857e2e5 None None] PATH_INFO: `/` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:26.059423 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 817/1633] 10.222.0.62 () {54 vars in 962 bytes} [Wed Aug 24 06:16:25 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:16:26.059423 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d223bc59-4259-45c6-aaa4-04408b407492 None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:26.059423 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d223bc59-4259-45c6-aaa4-04408b407492 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:26.061170 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d223bc59-4259-45c6-aaa4-04408b407492 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:26.061482 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3e920c-ea9c-4eb9-9e82-708173b8a740 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:16:26.061482 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd35808-10fb-46ed-86dc-ff62f79df3f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:16:26.061482 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd35808-10fb-46ed-86dc-ff62f79df3f5 admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:26.061482 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd35808-10fb-46ed-86dc-ff62f79df3f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:26.061864 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd35808-10fb-46ed-86dc-ff62f79df3f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:26.061864 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd35808-10fb-46ed-86dc-ff62f79df3f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:16:26.061864 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd35808-10fb-46ed-86dc-ff62f79df3f5 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:16:26.061864 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 817/1634] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Aug 24 06:16:25 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:16:26.061864 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd35808-10fb-46ed-86dc-ff62f79df3f5 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:16:26.061864 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd35808-10fb-46ed-86dc-ff62f79df3f5 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:16:26.063386 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3ba4a8-4f66-471a-995c-ed247cd6f8a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:16:26.063386 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3ba4a8-4f66-471a-995c-ed247cd6f8a8 admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:26.063386 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3ba4a8-4f66-471a-995c-ed247cd6f8a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:26.063386 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3ba4a8-4f66-471a-995c-ed247cd6f8a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:26.064408 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3ba4a8-4f66-471a-995c-ed247cd6f8a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:16:26.064408 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3ba4a8-4f66-471a-995c-ed247cd6f8a8 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:16:26.064819 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-d223bc59-4259-45c6-aaa4-04408b407492 None None] MFA Rules not processed for user `5d75f562443b4cd2a88aa37e268c8851`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:16:26.064819 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-d223bc59-4259-45c6-aaa4-04408b407492 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:16:26.065066 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 818/1635] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Aug 24 06:16:25 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:16:26.065066 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b193d5e3-d7e6-4579-aa02-23c6ca56a8a9 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:26.065066 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b193d5e3-d7e6-4579-aa02-23c6ca56a8a9 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:26.065066 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b193d5e3-d7e6-4579-aa02-23c6ca56a8a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:26.065066 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-b193d5e3-d7e6-4579-aa02-23c6ca56a8a9 None None] MFA Rules not processed for user `fb6d3743340241f98cacb9e0d063298c`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:16:26.065066 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-b193d5e3-d7e6-4579-aa02-23c6ca56a8a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:16:26.107387 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 819/1636] 10.222.0.62 () {58 vars in 1059 bytes} [Wed Aug 24 06:16:25 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:16:26.108628 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b193d5e3-d7e6-4579-aa02-23c6ca56a8a9 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:16:26.122487 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b193d5e3-d7e6-4579-aa02-23c6ca56a8a9 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:16:26.123216 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:16:26.129147 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5733575a-59ef-4aee-bec8-b3f37ecdad08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:16:26.129828 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5733575a-59ef-4aee-bec8-b3f37ecdad08 admin admin] REQUEST_METHOD: `PUT` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:26.130106 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5733575a-59ef-4aee-bec8-b3f37ecdad08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:26.130378 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5733575a-59ef-4aee-bec8-b3f37ecdad08 admin admin] PATH_INFO: `/v3/projects/b48751354827457e8ba7f0e71362f0de/users/16d019e12c164a208003ba5710c7ea15/roles/49e47cd157d743f6905f1db07ec878a7` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:26.132349 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5733575a-59ef-4aee-bec8-b3f37ecdad08 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b48751354827457e8ba7f0e71362f0de, user_id=16d019e12c164a208003ba5710c7ea15, role_id=49e47cd157d743f6905f1db07ec878a7)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:16:26.135114 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5733575a-59ef-4aee-bec8-b3f37ecdad08 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:16:26.143474 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 818/1637] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:16:25 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:16:26.145230 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d223bc59-4259-45c6-aaa4-04408b407492 None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:16:26.153939 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 820/1638] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Aug 24 06:16:26 2022] PUT /identity/v3/projects/b48751354827457e8ba7f0e71362f0de/users/16d019e12c164a208003ba5710c7ea15/roles/49e47cd157d743f6905f1db07ec878a7 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:16:26.155251 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d54d73-4fcf-4dda-bac5-8c2e45776bc3 None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:26.156065 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d54d73-4fcf-4dda-bac5-8c2e45776bc3 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:26.156065 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d54d73-4fcf-4dda-bac5-8c2e45776bc3 None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:26.166361 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 821/1639] 10.222.0.62 () {54 vars in 912 bytes} [Wed Aug 24 06:16:26 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:16:26.169858 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d223bc59-4259-45c6-aaa4-04408b407492 None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:16:26.171252 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d54d73-4fcf-4dda-bac5-8c2e45776bc3 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:16:26.243496 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d54d73-4fcf-4dda-bac5-8c2e45776bc3 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:16:26.247866 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6214128e-5866-48d5-9d82-621fbd23a48e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c4def55ac0e48b29836bc5c64bc7402', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:16:26.248484 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6214128e-5866-48d5-9d82-621fbd23a48e service cinder] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:26.248839 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6214128e-5866-48d5-9d82-621fbd23a48e service cinder] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:26.249117 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6214128e-5866-48d5-9d82-621fbd23a48e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:26.294676 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d87598-b475-4187-b25e-5c516b22d85b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:16:26.295365 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d87598-b475-4187-b25e-5c516b22d85b admin admin] REQUEST_METHOD: `PUT` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:26.295650 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d87598-b475-4187-b25e-5c516b22d85b admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:26.295931 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d87598-b475-4187-b25e-5c516b22d85b admin admin] PATH_INFO: `/v3/projects/889ae5dfed77476a834dade5174d751b/users/37c7468a5a3641999d6cb87d62394b3f/roles/49e47cd157d743f6905f1db07ec878a7` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:26.324750 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d87598-b475-4187-b25e-5c516b22d85b admin admin] RBAC: Authorizing `identity:create_grant(project_id=889ae5dfed77476a834dade5174d751b, user_id=37c7468a5a3641999d6cb87d62394b3f, role_id=49e47cd157d743f6905f1db07ec878a7)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:16:26.328081 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d87598-b475-4187-b25e-5c516b22d85b admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:16:26.343220 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6214128e-5866-48d5-9d82-621fbd23a48e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:16:26.344863 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6214128e-5866-48d5-9d82-621fbd23a48e service cinder] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:16:26.366701 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 822/1640] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:16:26 2022] GET /identity/v3/auth/tokens => generated 3345 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:16:26.381335 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5e9a2b-7c1a-4b38-b2d8-77cd7a472818 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:26.381814 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5e9a2b-7c1a-4b38-b2d8-77cd7a472818 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:26.382214 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5e9a2b-7c1a-4b38-b2d8-77cd7a472818 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:26.424814 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 819/1641] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Aug 24 06:16:26 2022] PUT /identity/v3/projects/889ae5dfed77476a834dade5174d751b/users/37c7468a5a3641999d6cb87d62394b3f/roles/49e47cd157d743f6905f1db07ec878a7 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:16:26.426458 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733bcc9c-3018-4230-8e26-b22a80e2d9ba None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:26.426746 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733bcc9c-3018-4230-8e26-b22a80e2d9ba None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:26.427021 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733bcc9c-3018-4230-8e26-b22a80e2d9ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:26.455526 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-8e5e9a2b-7c1a-4b38-b2d8-77cd7a472818 None None] MFA Rules not processed for user `16d019e12c164a208003ba5710c7ea15`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:16:26.458256 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-8e5e9a2b-7c1a-4b38-b2d8-77cd7a472818 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:16:26.460799 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-733bcc9c-3018-4230-8e26-b22a80e2d9ba None None] MFA Rules not processed for user `fb6d3743340241f98cacb9e0d063298c`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:16:26.463428 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-733bcc9c-3018-4230-8e26-b22a80e2d9ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:16:26.528124 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:16:26.608733 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 823/1642] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:16:26 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:16:26.624316 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5e9a2b-7c1a-4b38-b2d8-77cd7a472818 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:16:26.692342 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 820/1643] 10.222.0.62 () {58 vars in 1009 bytes} [Wed Aug 24 06:16:26 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:16:26.692342 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733bcc9c-3018-4230-8e26-b22a80e2d9ba None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:16:26.746323 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5e9a2b-7c1a-4b38-b2d8-77cd7a472818 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:16:26.751270 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61dab465-00a6-4476-953f-a48f8f81afba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:16:26.752359 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dab465-00a6-4476-953f-a48f8f81afba admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:26.752668 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dab465-00a6-4476-953f-a48f8f81afba admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:26.753252 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dab465-00a6-4476-953f-a48f8f81afba admin admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:26.754953 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61dab465-00a6-4476-953f-a48f8f81afba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:16:26.756763 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61dab465-00a6-4476-953f-a48f8f81afba admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:16:26.778151 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 824/1644] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Aug 24 06:16:26 2022] GET /identity/v3/roles => generated 1420 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:16:26.780979 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61dab465-00a6-4476-953f-a48f8f81afba admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:16:26.800940 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61dab465-00a6-4476-953f-a48f8f81afba admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:16:26.813939 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733bcc9c-3018-4230-8e26-b22a80e2d9ba None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:16:26.814105 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97837c99-53d3-4847-8331-7c4558038d10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:16:26.814539 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97837c99-53d3-4847-8331-7c4558038d10 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:26.814805 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97837c99-53d3-4847-8331-7c4558038d10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:26.815078 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97837c99-53d3-4847-8331-7c4558038d10 admin admin] PATH_INFO: `/v3/users/dbcbc15f5edb41d4bf58299ea45a62bd` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:26.817027 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97837c99-53d3-4847-8331-7c4558038d10 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dbcbc15f5edb41d4bf58299ea45a62bd)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:16:26.818689 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: WARNING py.warnings [None req-97837c99-53d3-4847-8331-7c4558038d10 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 06:16:26.818689 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: warnings.warn(msg) Aug 24 06:16:26.818689 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: Aug 24 06:16:26.819915 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97837c99-53d3-4847-8331-7c4558038d10 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:16:26.820712 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1abe80-1365-4d8a-8e96-b18d3a60074d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c4def55ac0e48b29836bc5c64bc7402', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:16:26.821641 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1abe80-1365-4d8a-8e96-b18d3a60074d service cinder] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:26.822278 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1abe80-1365-4d8a-8e96-b18d3a60074d service cinder] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:26.822854 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1abe80-1365-4d8a-8e96-b18d3a60074d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:26.836943 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1abe80-1365-4d8a-8e96-b18d3a60074d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:16:26.839327 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1abe80-1365-4d8a-8e96-b18d3a60074d service cinder] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:16:26.854878 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 821/1645] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:16:26 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:16:26.856179 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1abe80-1365-4d8a-8e96-b18d3a60074d service cinder] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:16:26.867381 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1abe80-1365-4d8a-8e96-b18d3a60074d service cinder] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:16:26.886632 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8f1934-1f7c-446f-9bd6-189fb874a9db service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c4def55ac0e48b29836bc5c64bc7402', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:16:26.887273 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8f1934-1f7c-446f-9bd6-189fb874a9db service cinder] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:26.887566 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8f1934-1f7c-446f-9bd6-189fb874a9db service cinder] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:26.887837 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8f1934-1f7c-446f-9bd6-189fb874a9db service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:26.906798 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8f1934-1f7c-446f-9bd6-189fb874a9db service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:16:26.908660 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8f1934-1f7c-446f-9bd6-189fb874a9db service cinder] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:16:26.936720 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 822/1646] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:16:26 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:16:26.939940 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8f1934-1f7c-446f-9bd6-189fb874a9db service cinder] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:16:26.966185 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8f1934-1f7c-446f-9bd6-189fb874a9db service cinder] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:16:26.980104 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b158e83-f0c7-414b-a7c7-787d76d911fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:16:26.981382 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b158e83-f0c7-414b-a7c7-787d76d911fa admin admin] REQUEST_METHOD: `PUT` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:26.981802 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b158e83-f0c7-414b-a7c7-787d76d911fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:26.982355 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b158e83-f0c7-414b-a7c7-787d76d911fa admin admin] PATH_INFO: `/v3/projects/889ae5dfed77476a834dade5174d751b/users/37c7468a5a3641999d6cb87d62394b3f/roles/49e47cd157d743f6905f1db07ec878a7` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:26.988734 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b158e83-f0c7-414b-a7c7-787d76d911fa admin admin] RBAC: Authorizing `identity:create_grant(project_id=889ae5dfed77476a834dade5174d751b, user_id=37c7468a5a3641999d6cb87d62394b3f, role_id=49e47cd157d743f6905f1db07ec878a7)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:16:27.000880 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b158e83-f0c7-414b-a7c7-787d76d911fa admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:16:27.020227 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 823/1647] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Aug 24 06:16:26 2022] PUT /identity/v3/projects/889ae5dfed77476a834dade5174d751b/users/37c7468a5a3641999d6cb87d62394b3f/roles/49e47cd157d743f6905f1db07ec878a7 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:16:27.150735 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-97837c99-53d3-4847-8331-7c4558038d10 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dbcbc15f5edb41d4bf58299ea45a62bd'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:16:27.167975 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-97837c99-53d3-4847-8331-7c4558038d10 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dbcbc15f5edb41d4bf58299ea45a62bd'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:16:27.170565 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-97837c99-53d3-4847-8331-7c4558038d10 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dbcbc15f5edb41d4bf58299ea45a62bd'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:16:27.193850 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930d48b6-2141-499a-861f-04d2a01f8eb9 None None] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:27.194377 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930d48b6-2141-499a-861f-04d2a01f8eb9 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:27.194786 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930d48b6-2141-499a-861f-04d2a01f8eb9 None None] PATH_INFO: `/` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:27.206193 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 824/1648] 10.222.0.62 () {54 vars in 912 bytes} [Wed Aug 24 06:16:27 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:16:27.206193 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8bdf5f-1d6c-48eb-9362-65c52bcce5ba None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:27.206193 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8bdf5f-1d6c-48eb-9362-65c52bcce5ba None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:27.206193 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8bdf5f-1d6c-48eb-9362-65c52bcce5ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:27.229646 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-5c8bdf5f-1d6c-48eb-9362-65c52bcce5ba None None] MFA Rules not processed for user `fb6d3743340241f98cacb9e0d063298c`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:16:27.231040 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-5c8bdf5f-1d6c-48eb-9362-65c52bcce5ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:16:27.287361 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-97837c99-53d3-4847-8331-7c4558038d10 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dbcbc15f5edb41d4bf58299ea45a62bd'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:16:27.334819 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-97837c99-53d3-4847-8331-7c4558038d10 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dbcbc15f5edb41d4bf58299ea45a62bd'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:16:27.357388 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 825/1649] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Aug 24 06:16:26 2022] DELETE /identity/v3/users/dbcbc15f5edb41d4bf58299ea45a62bd => generated 0 bytes in 576 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:16:27.357388 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97837c99-53d3-4847-8331-7c4558038d10 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:16:27.358828 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-97837c99-53d3-4847-8331-7c4558038d10 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:16:27.359483 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 825/1650] 10.222.0.62 () {58 vars in 1009 bytes} [Wed Aug 24 06:16:27 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 06:16:27.360867 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13621609-56e7-4f95-b324-cb10bdc10a4d None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:27.361158 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13621609-56e7-4f95-b324-cb10bdc10a4d None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:27.362420 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13621609-56e7-4f95-b324-cb10bdc10a4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:27.403433 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-13621609-56e7-4f95-b324-cb10bdc10a4d None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:16:27.404719 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-13621609-56e7-4f95-b324-cb10bdc10a4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:16:27.517733 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97837c99-53d3-4847-8331-7c4558038d10 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:16:27.526048 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 826/1651] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:16:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:16:27.526934 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13621609-56e7-4f95-b324-cb10bdc10a4d None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:16:27.529472 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d36fe4-91db-484e-b165-5337da06c429 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c4def55ac0e48b29836bc5c64bc7402', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:16:27.531010 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d36fe4-91db-484e-b165-5337da06c429 service cinder] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:27.531296 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d36fe4-91db-484e-b165-5337da06c429 service cinder] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:27.531563 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d36fe4-91db-484e-b165-5337da06c429 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:27.533246 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-13621609-56e7-4f95-b324-cb10bdc10a4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:16:27.533714 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-d6d36fe4-91db-484e-b165-5337da06c429 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:16:27.541645 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:16:27.559931 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13621609-56e7-4f95-b324-cb10bdc10a4d None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:16:27.567307 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ebe924-cae1-4cd0-b1a7-91c2d289b3c9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c4def55ac0e48b29836bc5c64bc7402', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:16:27.569161 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ebe924-cae1-4cd0-b1a7-91c2d289b3c9 service cinder] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:27.569741 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ebe924-cae1-4cd0-b1a7-91c2d289b3c9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:27.570048 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ebe924-cae1-4cd0-b1a7-91c2d289b3c9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:27.683288 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d36fe4-91db-484e-b165-5337da06c429 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:16:27.684790 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d36fe4-91db-484e-b165-5337da06c429 service cinder] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:16:27.712447 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 826/1652] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:16:27 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:16:27.714232 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d36fe4-91db-484e-b165-5337da06c429 service cinder] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:16:27.717264 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-d6d36fe4-91db-484e-b165-5337da06c429 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:16:27.750140 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ebe924-cae1-4cd0-b1a7-91c2d289b3c9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:16:27.754606 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ebe924-cae1-4cd0-b1a7-91c2d289b3c9 service cinder] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:16:27.808759 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 827/1653] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:16:27 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 272 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:16:27.845387 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d36fe4-91db-484e-b165-5337da06c429 service cinder] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:16:27.850948 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3885e73c-9035-498b-9d42-82b86fb33cfe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bb6c79db75e4d169331c02133d1ec32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:16:27.851576 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3885e73c-9035-498b-9d42-82b86fb33cfe service neutron] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:27.851984 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3885e73c-9035-498b-9d42-82b86fb33cfe service neutron] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:27.852269 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3885e73c-9035-498b-9d42-82b86fb33cfe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:27.865291 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3885e73c-9035-498b-9d42-82b86fb33cfe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:16:27.869849 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3885e73c-9035-498b-9d42-82b86fb33cfe service neutron] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:16:27.882238 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 827/1654] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:16:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:16:27.888571 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:16:28.043089 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ebe924-cae1-4cd0-b1a7-91c2d289b3c9 service cinder] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:16:28.045214 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-a0ebe924-cae1-4cd0-b1a7-91c2d289b3c9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:16:28.074996 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ebe924-cae1-4cd0-b1a7-91c2d289b3c9 service cinder] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:16:28.092828 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1276c7c-77a0-4a92-9d49-1be0ae3cfea7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:16:28.094093 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1276c7c-77a0-4a92-9d49-1be0ae3cfea7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:28.094571 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1276c7c-77a0-4a92-9d49-1be0ae3cfea7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:28.095085 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1276c7c-77a0-4a92-9d49-1be0ae3cfea7 admin admin] PATH_INFO: `/v3/users/43ca918762024b82b73a54393a384533` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:28.096615 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1276c7c-77a0-4a92-9d49-1be0ae3cfea7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=43ca918762024b82b73a54393a384533)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:16:28.100855 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1276c7c-77a0-4a92-9d49-1be0ae3cfea7 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:16:28.395491 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-f1276c7c-77a0-4a92-9d49-1be0ae3cfea7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '43ca918762024b82b73a54393a384533'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:16:28.403913 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-f1276c7c-77a0-4a92-9d49-1be0ae3cfea7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '43ca918762024b82b73a54393a384533'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:16:28.415838 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-f1276c7c-77a0-4a92-9d49-1be0ae3cfea7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '43ca918762024b82b73a54393a384533'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:16:28.416709 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-f1276c7c-77a0-4a92-9d49-1be0ae3cfea7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '43ca918762024b82b73a54393a384533'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:16:28.528423 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-f1276c7c-77a0-4a92-9d49-1be0ae3cfea7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '43ca918762024b82b73a54393a384533'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:16:28.570460 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 828/1655] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Aug 24 06:16:28 2022] DELETE /identity/v3/users/43ca918762024b82b73a54393a384533 => generated 0 bytes in 528 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:16:28.574551 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ec914b-4519-4e13-be52-9ed4b8d419f2 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:28.575053 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ec914b-4519-4e13-be52-9ed4b8d419f2 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:28.575122 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ec914b-4519-4e13-be52-9ed4b8d419f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:28.618117 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-32ec914b-4519-4e13-be52-9ed4b8d419f2 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:16:28.619521 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-32ec914b-4519-4e13-be52-9ed4b8d419f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:16:28.760091 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 828/1656] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:16:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:16:28.790343 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1276c7c-77a0-4a92-9d49-1be0ae3cfea7 admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:16:28.794491 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-f1276c7c-77a0-4a92-9d49-1be0ae3cfea7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:16:28.912393 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1276c7c-77a0-4a92-9d49-1be0ae3cfea7 admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:16:28.917956 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9722e5b-edc3-42eb-89a1-1de8feb8ca8a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bb6c79db75e4d169331c02133d1ec32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:16:28.918595 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9722e5b-edc3-42eb-89a1-1de8feb8ca8a service neutron] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:28.918870 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9722e5b-edc3-42eb-89a1-1de8feb8ca8a service neutron] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:28.919142 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9722e5b-edc3-42eb-89a1-1de8feb8ca8a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:28.945079 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9722e5b-edc3-42eb-89a1-1de8feb8ca8a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:16:28.947083 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9722e5b-edc3-42eb-89a1-1de8feb8ca8a service neutron] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:16:28.968600 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 829/1657] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:16:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:16:29.090050 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:16:29.577951 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ec914b-4519-4e13-be52-9ed4b8d419f2 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:16:29.579405 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-32ec914b-4519-4e13-be52-9ed4b8d419f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:16:29.606256 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ec914b-4519-4e13-be52-9ed4b8d419f2 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:16:29.620386 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226563af-f0ec-4b4b-bcc2-1f8dd3d6f418 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:16:29.621088 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226563af-f0ec-4b4b-bcc2-1f8dd3d6f418 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:29.621541 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226563af-f0ec-4b4b-bcc2-1f8dd3d6f418 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:29.621956 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226563af-f0ec-4b4b-bcc2-1f8dd3d6f418 admin admin] PATH_INFO: `/v3/projects/038bd6c34b4b4881a5b9517b458b2c50` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:29.623668 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226563af-f0ec-4b4b-bcc2-1f8dd3d6f418 admin admin] RBAC: Authorizing `identity:delete_project(project_id=038bd6c34b4b4881a5b9517b458b2c50)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:16:29.627872 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226563af-f0ec-4b4b-bcc2-1f8dd3d6f418 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:16:29.855136 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-226563af-f0ec-4b4b-bcc2-1f8dd3d6f418 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '038bd6c34b4b4881a5b9517b458b2c50', 'request_id': 'req-226563af-f0ec-4b4b-bcc2-1f8dd3d6f418'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:16:29.859969 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:16:29.870678 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-226563af-f0ec-4b4b-bcc2-1f8dd3d6f418 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '038bd6c34b4b4881a5b9517b458b2c50', 'request_id': 'req-226563af-f0ec-4b4b-bcc2-1f8dd3d6f418'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:16:29.893271 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-226563af-f0ec-4b4b-bcc2-1f8dd3d6f418 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '038bd6c34b4b4881a5b9517b458b2c50', 'request_id': 'req-226563af-f0ec-4b4b-bcc2-1f8dd3d6f418'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:16:29.979792 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-226563af-f0ec-4b4b-bcc2-1f8dd3d6f418 admin admin] The token cache is being invalidate because project 038bd6c34b4b4881a5b9517b458b2c50 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81852) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:16:29.980088 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-226563af-f0ec-4b4b-bcc2-1f8dd3d6f418 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:16:29.989741 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-226563af-f0ec-4b4b-bcc2-1f8dd3d6f418 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:16:29.993769 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 829/1658] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Aug 24 06:16:29 2022] DELETE /identity/v3/projects/038bd6c34b4b4881a5b9517b458b2c50 => generated 0 bytes in 413 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:16:29.995417 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9722e5b-edc3-42eb-89a1-1de8feb8ca8a service neutron] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:16:30.012701 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9722e5b-edc3-42eb-89a1-1de8feb8ca8a service neutron] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:16:30.185129 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f31b675-ba65-4396-9571-f3ebab9c37b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:16:30.188161 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f31b675-ba65-4396-9571-f3ebab9c37b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:30.188540 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f31b675-ba65-4396-9571-f3ebab9c37b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:30.188971 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f31b675-ba65-4396-9571-f3ebab9c37b6 admin admin] PATH_INFO: `/v3/users/5d75f562443b4cd2a88aa37e268c8851` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:30.194367 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f31b675-ba65-4396-9571-f3ebab9c37b6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5d75f562443b4cd2a88aa37e268c8851)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:16:30.207824 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f31b675-ba65-4396-9571-f3ebab9c37b6 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:16:30.471737 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-2f31b675-ba65-4396-9571-f3ebab9c37b6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5d75f562443b4cd2a88aa37e268c8851'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:16:30.481060 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-2f31b675-ba65-4396-9571-f3ebab9c37b6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5d75f562443b4cd2a88aa37e268c8851'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:16:30.481829 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-2f31b675-ba65-4396-9571-f3ebab9c37b6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5d75f562443b4cd2a88aa37e268c8851'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:16:30.482669 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-2f31b675-ba65-4396-9571-f3ebab9c37b6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5d75f562443b4cd2a88aa37e268c8851'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:16:30.557104 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-2f31b675-ba65-4396-9571-f3ebab9c37b6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5d75f562443b4cd2a88aa37e268c8851'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:16:30.609313 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 830/1659] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Aug 24 06:16:29 2022] DELETE /identity/v3/users/5d75f562443b4cd2a88aa37e268c8851 => generated 0 bytes in 614 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:16:30.642846 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:16:31.062191 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226563af-f0ec-4b4b-bcc2-1f8dd3d6f418 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:16:31.064388 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-226563af-f0ec-4b4b-bcc2-1f8dd3d6f418 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:16:31.088456 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:16:31.091724 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226563af-f0ec-4b4b-bcc2-1f8dd3d6f418 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:16:31.299147 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f20569-52d5-46af-bae7-bbed49441d75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:16:31.300174 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f20569-52d5-46af-bae7-bbed49441d75 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:31.300599 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f20569-52d5-46af-bae7-bbed49441d75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:31.300885 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f20569-52d5-46af-bae7-bbed49441d75 admin admin] PATH_INFO: `/v3/projects/98863eced91a48beba0ea96d23fb851d` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:31.302226 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f20569-52d5-46af-bae7-bbed49441d75 admin admin] RBAC: Authorizing `identity:delete_project(project_id=98863eced91a48beba0ea96d23fb851d)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:16:31.304248 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f20569-52d5-46af-bae7-bbed49441d75 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:16:31.368313 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b2355a-3a9f-4630-a830-902889a295d5 None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:31.368928 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b2355a-3a9f-4630-a830-902889a295d5 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:31.369216 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b2355a-3a9f-4630-a830-902889a295d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:31.452419 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-b1b2355a-3a9f-4630-a830-902889a295d5 None None] MFA Rules not processed for user `25908fb02f7f46f5b9e483811302a67e`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:16:31.455314 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-b1b2355a-3a9f-4630-a830-902889a295d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:16:31.469379 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-a8f20569-52d5-46af-bae7-bbed49441d75 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '98863eced91a48beba0ea96d23fb851d', 'request_id': 'req-a8f20569-52d5-46af-bae7-bbed49441d75'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:16:31.477330 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-a8f20569-52d5-46af-bae7-bbed49441d75 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '98863eced91a48beba0ea96d23fb851d', 'request_id': 'req-a8f20569-52d5-46af-bae7-bbed49441d75'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:16:31.486875 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-a8f20569-52d5-46af-bae7-bbed49441d75 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '98863eced91a48beba0ea96d23fb851d', 'request_id': 'req-a8f20569-52d5-46af-bae7-bbed49441d75'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:16:31.609879 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-a8f20569-52d5-46af-bae7-bbed49441d75 admin admin] The token cache is being invalidate because project 98863eced91a48beba0ea96d23fb851d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81852) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:16:31.610382 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-a8f20569-52d5-46af-bae7-bbed49441d75 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:16:31.611202 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-a8f20569-52d5-46af-bae7-bbed49441d75 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:16:31.615728 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 830/1660] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Aug 24 06:16:31 2022] DELETE /identity/v3/projects/98863eced91a48beba0ea96d23fb851d => generated 0 bytes in 553 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:16:31.620003 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473d8367-d50b-417f-b852-6291975aa43b None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:31.620673 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473d8367-d50b-417f-b852-6291975aa43b None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:31.622437 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473d8367-d50b-417f-b852-6291975aa43b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:31.656692 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 831/1661] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:16:31 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 287 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:16:31.670612 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b2355a-3a9f-4630-a830-902889a295d5 None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:16:31.672098 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-b1b2355a-3a9f-4630-a830-902889a295d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:16:31.683009 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-473d8367-d50b-417f-b852-6291975aa43b None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:16:31.686959 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-473d8367-d50b-417f-b852-6291975aa43b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:16:31.785434 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 831/1662] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:16:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:16:31.800625 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8786fb1c-bb42-4a0b-8cfc-bff6dc7750c4 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:31.801004 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8786fb1c-bb42-4a0b-8cfc-bff6dc7750c4 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:31.802242 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8786fb1c-bb42-4a0b-8cfc-bff6dc7750c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:31.838375 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-8786fb1c-bb42-4a0b-8cfc-bff6dc7750c4 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:16:31.839926 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-8786fb1c-bb42-4a0b-8cfc-bff6dc7750c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:16:31.853754 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 832/1663] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:16:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:16:31.856198 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8786fb1c-bb42-4a0b-8cfc-bff6dc7750c4 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:16:31.870525 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8786fb1c-bb42-4a0b-8cfc-bff6dc7750c4 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:16:31.873461 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b2355a-3a9f-4630-a830-902889a295d5 None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:16:31.876408 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f89375d-023a-4c43-91cf-db97d336359b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:16:31.879370 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f89375d-023a-4c43-91cf-db97d336359b admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:31.879844 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f89375d-023a-4c43-91cf-db97d336359b admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:31.880306 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f89375d-023a-4c43-91cf-db97d336359b admin admin] PATH_INFO: `/v3/domains` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:31.881249 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f89375d-023a-4c43-91cf-db97d336359b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:16:31.881964 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f89375d-023a-4c43-91cf-db97d336359b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:16:31.885625 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f89375d-023a-4c43-91cf-db97d336359b admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:16:31.909634 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d5cbf7-7fe8-4d4c-ba4b-84f0839a7188 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb6d3743340241f98cacb9e0d063298c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:16:31.910956 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d5cbf7-7fe8-4d4c-ba4b-84f0839a7188 service nova] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:31.911366 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d5cbf7-7fe8-4d4c-ba4b-84f0839a7188 service nova] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:31.911640 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d5cbf7-7fe8-4d4c-ba4b-84f0839a7188 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:31.917525 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 833/1664] 10.222.0.62 () {58 vars in 1153 bytes} [Wed Aug 24 06:16:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:16:31.918823 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f89375d-023a-4c43-91cf-db97d336359b admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:16:31.933932 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d5cbf7-7fe8-4d4c-ba4b-84f0839a7188 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:16:31.934904 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f89375d-023a-4c43-91cf-db97d336359b admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:16:31.939516 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d5cbf7-7fe8-4d4c-ba4b-84f0839a7188 service nova] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:16:31.943587 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af9fabf-796e-4daf-a88f-26140c9032f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:16:31.945413 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af9fabf-796e-4daf-a88f-26140c9032f9 admin admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:31.950897 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af9fabf-796e-4daf-a88f-26140c9032f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:31.951388 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af9fabf-796e-4daf-a88f-26140c9032f9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:31.952285 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af9fabf-796e-4daf-a88f-26140c9032f9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:16:31.955882 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af9fabf-796e-4daf-a88f-26140c9032f9 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:16:31.963145 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 832/1665] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:16:31 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:16:32.039378 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db4285f-651b-4f85-812e-70817fa92308 None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:32.039681 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db4285f-651b-4f85-812e-70817fa92308 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:32.040931 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db4285f-651b-4f85-812e-70817fa92308 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:32.116037 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 834/1666] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Aug 24 06:16:31 2022] POST /identity/v3/projects => generated 355 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:16:32.116211 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-7db4285f-651b-4f85-812e-70817fa92308 None None] MFA Rules not processed for user `37c7468a5a3641999d6cb87d62394b3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:16:32.118008 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af9fabf-796e-4daf-a88f-26140c9032f9 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:16:32.118084 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-7db4285f-651b-4f85-812e-70817fa92308 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:16:32.243125 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 833/1667] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:16:32 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:16:32.249345 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af9fabf-796e-4daf-a88f-26140c9032f9 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:16:32.259464 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8d6ff7-4d92-45f9-85b2-a6e98f0d046a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:16:32.260103 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8d6ff7-4d92-45f9-85b2-a6e98f0d046a admin admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:32.260386 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8d6ff7-4d92-45f9-85b2-a6e98f0d046a admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:32.260833 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8d6ff7-4d92-45f9-85b2-a6e98f0d046a admin admin] PATH_INFO: `/v3/users` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:32.261347 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8d6ff7-4d92-45f9-85b2-a6e98f0d046a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:16:32.263822 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8d6ff7-4d92-45f9-85b2-a6e98f0d046a admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:16:32.328689 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:16:32.374871 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 835/1668] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Aug 24 06:16:32 2022] POST /identity/v3/users => generated 278 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:16:32.378360 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db4285f-651b-4f85-812e-70817fa92308 None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:16:32.391986 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db4285f-651b-4f85-812e-70817fa92308 None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:16:32.404543 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6469d99f-6930-4773-b4d8-af5dd1771673 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:16:32.405167 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6469d99f-6930-4773-b4d8-af5dd1771673 admin admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:32.405447 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6469d99f-6930-4773-b4d8-af5dd1771673 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:32.405720 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6469d99f-6930-4773-b4d8-af5dd1771673 admin admin] PATH_INFO: `/v3/roles` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:32.406370 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6469d99f-6930-4773-b4d8-af5dd1771673 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:16:32.411820 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6469d99f-6930-4773-b4d8-af5dd1771673 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:16:32.424326 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 834/1669] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Aug 24 06:16:32 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:16:32.429252 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8d6ff7-4d92-45f9-85b2-a6e98f0d046a admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:16:32.444718 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8d6ff7-4d92-45f9-85b2-a6e98f0d046a admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:16:32.449628 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b062688b-08a0-4c6e-b003-2941f1710c55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:16:32.450678 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b062688b-08a0-4c6e-b003-2941f1710c55 admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:32.450678 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b062688b-08a0-4c6e-b003-2941f1710c55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:32.450843 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b062688b-08a0-4c6e-b003-2941f1710c55 admin admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:32.451706 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b062688b-08a0-4c6e-b003-2941f1710c55 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:16:32.453663 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b062688b-08a0-4c6e-b003-2941f1710c55 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:16:32.471139 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 836/1670] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Aug 24 06:16:32 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:16:32.473439 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6469d99f-6930-4773-b4d8-af5dd1771673 admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:16:32.486109 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6469d99f-6930-4773-b4d8-af5dd1771673 admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:16:32.507911 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce4a96d-7c12-41e9-acfd-e9bca89f589e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:16:32.511672 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce4a96d-7c12-41e9-acfd-e9bca89f589e admin admin] REQUEST_METHOD: `PUT` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:32.512212 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce4a96d-7c12-41e9-acfd-e9bca89f589e admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:32.520032 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce4a96d-7c12-41e9-acfd-e9bca89f589e admin admin] PATH_INFO: `/v3/projects/32898113fdce4e8987562cc754c585d5/users/e3b7ebadfed24f77895403aa56e4ecaa/roles/580aa7dbb38d4cff8f9d1bb8983f99ef` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:32.546167 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce4a96d-7c12-41e9-acfd-e9bca89f589e admin admin] RBAC: Authorizing `identity:create_grant(project_id=32898113fdce4e8987562cc754c585d5, user_id=e3b7ebadfed24f77895403aa56e4ecaa, role_id=580aa7dbb38d4cff8f9d1bb8983f99ef)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:16:32.551076 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce4a96d-7c12-41e9-acfd-e9bca89f589e admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:16:32.641676 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b062688b-08a0-4c6e-b003-2941f1710c55 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:16:32.643445 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-b062688b-08a0-4c6e-b003-2941f1710c55 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:16:32.680566 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 835/1671] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Aug 24 06:16:32 2022] PUT /identity/v3/projects/32898113fdce4e8987562cc754c585d5/users/e3b7ebadfed24f77895403aa56e4ecaa/roles/580aa7dbb38d4cff8f9d1bb8983f99ef => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:16:32.682379 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d524320-64f2-4138-8de8-0a35ff1b5190 None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:32.683628 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d524320-64f2-4138-8de8-0a35ff1b5190 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:32.683939 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d524320-64f2-4138-8de8-0a35ff1b5190 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:32.685523 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b062688b-08a0-4c6e-b003-2941f1710c55 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:16:32.799783 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-6d524320-64f2-4138-8de8-0a35ff1b5190 None None] MFA Rules not processed for user `e3b7ebadfed24f77895403aa56e4ecaa`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:16:32.803278 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-6d524320-64f2-4138-8de8-0a35ff1b5190 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:16:32.841242 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f94c832-9a0c-406c-b458-9bd2f559f734 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:16:32.843682 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f94c832-9a0c-406c-b458-9bd2f559f734 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:32.843682 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f94c832-9a0c-406c-b458-9bd2f559f734 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:32.843682 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f94c832-9a0c-406c-b458-9bd2f559f734 admin admin] PATH_INFO: `/v3/projects/ce6c24088632410ba262a9e202c61fa8` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:32.848067 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f94c832-9a0c-406c-b458-9bd2f559f734 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ce6c24088632410ba262a9e202c61fa8)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:16:32.849968 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f94c832-9a0c-406c-b458-9bd2f559f734 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:16:32.922992 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:16:32.959463 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 836/1672] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:16:32 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 279 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:16:32.964987 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d524320-64f2-4138-8de8-0a35ff1b5190 None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:16:32.989933 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d524320-64f2-4138-8de8-0a35ff1b5190 None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:16:33.015427 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-8f94c832-9a0c-406c-b458-9bd2f559f734 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ce6c24088632410ba262a9e202c61fa8', 'request_id': 'req-8f94c832-9a0c-406c-b458-9bd2f559f734'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:16:33.037759 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-8f94c832-9a0c-406c-b458-9bd2f559f734 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ce6c24088632410ba262a9e202c61fa8', 'request_id': 'req-8f94c832-9a0c-406c-b458-9bd2f559f734'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:16:33.047252 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-8f94c832-9a0c-406c-b458-9bd2f559f734 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ce6c24088632410ba262a9e202c61fa8', 'request_id': 'req-8f94c832-9a0c-406c-b458-9bd2f559f734'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:16:33.122132 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92535f68-b589-475a-bf8c-9088948f7fc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:16:33.122870 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92535f68-b589-475a-bf8c-9088948f7fc1 admin admin] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:33.123338 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92535f68-b589-475a-bf8c-9088948f7fc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:33.123861 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92535f68-b589-475a-bf8c-9088948f7fc1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:33.124649 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92535f68-b589-475a-bf8c-9088948f7fc1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:16:33.126886 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92535f68-b589-475a-bf8c-9088948f7fc1 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:16:33.139853 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-8f94c832-9a0c-406c-b458-9bd2f559f734 admin admin] The token cache is being invalidate because project ce6c24088632410ba262a9e202c61fa8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81852) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:16:33.140156 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-8f94c832-9a0c-406c-b458-9bd2f559f734 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:16:33.141134 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-8f94c832-9a0c-406c-b458-9bd2f559f734 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:16:33.142724 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 837/1673] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Aug 24 06:16:32 2022] DELETE /identity/v3/projects/ce6c24088632410ba262a9e202c61fa8 => generated 0 bytes in 502 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:16:33.150004 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28accf1a-aa1d-41ef-bb93-1103ea3ac5da None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:33.150004 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28accf1a-aa1d-41ef-bb93-1103ea3ac5da None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:33.150004 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28accf1a-aa1d-41ef-bb93-1103ea3ac5da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:33.197737 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-28accf1a-aa1d-41ef-bb93-1103ea3ac5da None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:16:33.200115 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-28accf1a-aa1d-41ef-bb93-1103ea3ac5da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:16:33.236216 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 838/1674] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:16:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 06:16:33.249983 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43994f6-2d80-4516-b83f-653409a3db31 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:33.251604 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43994f6-2d80-4516-b83f-653409a3db31 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:33.251604 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43994f6-2d80-4516-b83f-653409a3db31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:33.293758 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-b43994f6-2d80-4516-b83f-653409a3db31 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:16:33.298395 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-b43994f6-2d80-4516-b83f-653409a3db31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:16:33.320662 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 839/1675] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:16:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:16:33.324276 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43994f6-2d80-4516-b83f-653409a3db31 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:16:33.339969 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43994f6-2d80-4516-b83f-653409a3db31 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:16:33.353482 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 837/1676] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Aug 24 06:16:32 2022] POST /identity/v3/projects => generated 357 bytes in 388 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 06:16:33.356189 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92535f68-b589-475a-bf8c-9088948f7fc1 admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:16:33.362557 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-92535f68-b589-475a-bf8c-9088948f7fc1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:16:33.489956 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d909c762-7171-4bdb-80c9-280bd7da3467 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:16:33.489956 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d909c762-7171-4bdb-80c9-280bd7da3467 admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:33.489956 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d909c762-7171-4bdb-80c9-280bd7da3467 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:33.490372 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d909c762-7171-4bdb-80c9-280bd7da3467 admin admin] PATH_INFO: `/v3/domains` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:33.492725 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d909c762-7171-4bdb-80c9-280bd7da3467 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:16:33.493062 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d909c762-7171-4bdb-80c9-280bd7da3467 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:16:33.494704 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d909c762-7171-4bdb-80c9-280bd7da3467 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:16:33.548847 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 840/1677] 10.222.0.62 () {58 vars in 1153 bytes} [Wed Aug 24 06:16:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:16:33.551231 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d909c762-7171-4bdb-80c9-280bd7da3467 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:16:33.551998 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92535f68-b589-475a-bf8c-9088948f7fc1 admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:16:33.566126 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83582994-beff-4eb2-a2e3-89218fe9fa63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:16:33.567437 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83582994-beff-4eb2-a2e3-89218fe9fa63 admin admin] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:33.568802 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83582994-beff-4eb2-a2e3-89218fe9fa63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:33.569486 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83582994-beff-4eb2-a2e3-89218fe9fa63 admin admin] PATH_INFO: `/v3/users` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:33.570413 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83582994-beff-4eb2-a2e3-89218fe9fa63 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:16:33.573263 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83582994-beff-4eb2-a2e3-89218fe9fa63 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:16:33.581471 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d909c762-7171-4bdb-80c9-280bd7da3467 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:16:33.590073 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa28b5a8-cab0-4c45-a970-b0355d122c15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:16:33.591808 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa28b5a8-cab0-4c45-a970-b0355d122c15 admin admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:33.592343 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa28b5a8-cab0-4c45-a970-b0355d122c15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:33.593308 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa28b5a8-cab0-4c45-a970-b0355d122c15 admin admin] PATH_INFO: `/v3/projects` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:33.594458 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa28b5a8-cab0-4c45-a970-b0355d122c15 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:16:33.596730 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa28b5a8-cab0-4c45-a970-b0355d122c15 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:16:33.695114 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 838/1678] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Aug 24 06:16:33 2022] POST /identity/v3/users => generated 285 bytes in 341 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:16:33.699001 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83582994-beff-4eb2-a2e3-89218fe9fa63 admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:16:33.717564 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83582994-beff-4eb2-a2e3-89218fe9fa63 admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:16:33.728287 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 841/1679] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Aug 24 06:16:33 2022] POST /identity/v3/projects => generated 365 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:16:33.730516 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa28b5a8-cab0-4c45-a970-b0355d122c15 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:16:33.733285 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90753f0b-8377-468a-983c-84420d732467 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:16:33.733554 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90753f0b-8377-468a-983c-84420d732467 admin admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:33.733872 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90753f0b-8377-468a-983c-84420d732467 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:33.734146 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90753f0b-8377-468a-983c-84420d732467 admin admin] PATH_INFO: `/v3/roles` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:33.734808 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90753f0b-8377-468a-983c-84420d732467 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:16:33.736429 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90753f0b-8377-468a-983c-84420d732467 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:16:33.761124 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 839/1680] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Aug 24 06:16:33 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 06:16:33.761828 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90753f0b-8377-468a-983c-84420d732467 admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:16:33.782148 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90753f0b-8377-468a-983c-84420d732467 admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:16:33.819941 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:16:33.874885 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa28b5a8-cab0-4c45-a970-b0355d122c15 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:16:33.886925 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0923602a-3bb0-4092-b99c-89632fff836a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:16:33.887337 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0923602a-3bb0-4092-b99c-89632fff836a admin admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:33.888803 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0923602a-3bb0-4092-b99c-89632fff836a admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:33.888803 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0923602a-3bb0-4092-b99c-89632fff836a admin admin] PATH_INFO: `/v3/users` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:33.888803 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0923602a-3bb0-4092-b99c-89632fff836a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:16:33.890457 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0923602a-3bb0-4092-b99c-89632fff836a admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:16:33.905402 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b1ac22-7543-48a2-96e8-7178556bc202 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:16:33.910526 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b1ac22-7543-48a2-96e8-7178556bc202 admin admin] REQUEST_METHOD: `PUT` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:33.910526 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b1ac22-7543-48a2-96e8-7178556bc202 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:33.910526 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b1ac22-7543-48a2-96e8-7178556bc202 admin admin] PATH_INFO: `/v3/projects/0c35d1486ae945e984ce36a962b1e523/users/385444b8b35a4ad2a3617d6b9f24878a/roles/49e47cd157d743f6905f1db07ec878a7` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:33.926567 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b1ac22-7543-48a2-96e8-7178556bc202 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c35d1486ae945e984ce36a962b1e523, user_id=385444b8b35a4ad2a3617d6b9f24878a, role_id=49e47cd157d743f6905f1db07ec878a7)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:16:33.930758 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b1ac22-7543-48a2-96e8-7178556bc202 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:16:34.008349 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 842/1681] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Aug 24 06:16:33 2022] POST /identity/v3/users => generated 283 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 06:16:34.011285 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0923602a-3bb0-4092-b99c-89632fff836a admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:16:34.023555 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 840/1682] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Aug 24 06:16:33 2022] PUT /identity/v3/projects/0c35d1486ae945e984ce36a962b1e523/users/385444b8b35a4ad2a3617d6b9f24878a/roles/49e47cd157d743f6905f1db07ec878a7 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:16:34.027461 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b1ac22-7543-48a2-96e8-7178556bc202 admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:16:34.035530 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0923602a-3bb0-4092-b99c-89632fff836a admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:16:34.171500 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764d65a5-ea16-4cc2-890d-0378186fbbb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:16:34.172297 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764d65a5-ea16-4cc2-890d-0378186fbbb4 admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:34.172703 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764d65a5-ea16-4cc2-890d-0378186fbbb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:34.173075 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764d65a5-ea16-4cc2-890d-0378186fbbb4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:34.173959 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764d65a5-ea16-4cc2-890d-0378186fbbb4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:16:34.177139 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764d65a5-ea16-4cc2-890d-0378186fbbb4 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:16:34.193007 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b1ac22-7543-48a2-96e8-7178556bc202 admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:16:34.199253 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 843/1683] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Aug 24 06:16:34 2022] GET /identity/v3/roles => generated 1420 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 06:16:34.202642 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764d65a5-ea16-4cc2-890d-0378186fbbb4 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:16:34.209301 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8293915f-a454-43ba-a40d-24078e9e6130 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:16:34.210213 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8293915f-a454-43ba-a40d-24078e9e6130 admin admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:34.210562 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8293915f-a454-43ba-a40d-24078e9e6130 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:34.210923 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8293915f-a454-43ba-a40d-24078e9e6130 admin admin] PATH_INFO: `/v3/roles` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:34.211652 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8293915f-a454-43ba-a40d-24078e9e6130 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:16:34.214893 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8293915f-a454-43ba-a40d-24078e9e6130 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:16:34.224903 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764d65a5-ea16-4cc2-890d-0378186fbbb4 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:16:34.237713 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 841/1684] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Aug 24 06:16:34 2022] GET /identity/v3/roles => generated 1420 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 06:16:34.240929 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd00fe9-edb7-45e8-b5e5-94410c4dd116 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:16:34.242636 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd00fe9-edb7-45e8-b5e5-94410c4dd116 admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:34.242636 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd00fe9-edb7-45e8-b5e5-94410c4dd116 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:34.242636 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd00fe9-edb7-45e8-b5e5-94410c4dd116 admin admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:34.242865 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd00fe9-edb7-45e8-b5e5-94410c4dd116 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:16:34.246763 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbbf71f-53bb-47aa-834c-fe1658b50457 None None] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:34.247058 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbbf71f-53bb-47aa-834c-fe1658b50457 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:34.247901 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd00fe9-edb7-45e8-b5e5-94410c4dd116 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:16:34.249590 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbbf71f-53bb-47aa-834c-fe1658b50457 None None] PATH_INFO: `/` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:34.251953 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 842/1685] 10.222.0.62 () {54 vars in 921 bytes} [Wed Aug 24 06:16:34 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:16:34.253696 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbbf71f-53bb-47aa-834c-fe1658b50457 None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:16:34.278044 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 844/1686] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Aug 24 06:16:34 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:16:34.278572 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4417dea2-0525-45bc-a70b-0f3affed8df6 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:34.278572 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4417dea2-0525-45bc-a70b-0f3affed8df6 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:34.278866 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4417dea2-0525-45bc-a70b-0f3affed8df6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:34.288269 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbbf71f-53bb-47aa-834c-fe1658b50457 None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:16:34.299482 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed5145b-4146-4e7b-a9c2-4cc68cc58506 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:16:34.301134 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed5145b-4146-4e7b-a9c2-4cc68cc58506 admin admin] REQUEST_METHOD: `PUT` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:34.301642 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed5145b-4146-4e7b-a9c2-4cc68cc58506 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:34.301911 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed5145b-4146-4e7b-a9c2-4cc68cc58506 admin admin] PATH_INFO: `/v3/projects/0c35d1486ae945e984ce36a962b1e523/users/385444b8b35a4ad2a3617d6b9f24878a/roles/49e47cd157d743f6905f1db07ec878a7` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:34.308606 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed5145b-4146-4e7b-a9c2-4cc68cc58506 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c35d1486ae945e984ce36a962b1e523, user_id=385444b8b35a4ad2a3617d6b9f24878a, role_id=49e47cd157d743f6905f1db07ec878a7)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:16:34.313500 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed5145b-4146-4e7b-a9c2-4cc68cc58506 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:16:34.325676 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-4417dea2-0525-45bc-a70b-0f3affed8df6 None None] MFA Rules not processed for user `abe9d8c9291845dfa4ef55034e5c9e32`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:16:34.331107 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-4417dea2-0525-45bc-a70b-0f3affed8df6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:16:34.346152 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 843/1687] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Aug 24 06:16:34 2022] PUT /identity/v3/projects/0c35d1486ae945e984ce36a962b1e523/users/385444b8b35a4ad2a3617d6b9f24878a/roles/49e47cd157d743f6905f1db07ec878a7 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:16:34.357421 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed5145b-4146-4e7b-a9c2-4cc68cc58506 admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:16:34.460754 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:16:34.499247 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 845/1688] 10.222.0.62 () {58 vars in 1018 bytes} [Wed Aug 24 06:16:34 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:16:34.501463 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154cdbf4-6631-4486-a547-3a3d62870695 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:34.502315 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154cdbf4-6631-4486-a547-3a3d62870695 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:34.504300 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154cdbf4-6631-4486-a547-3a3d62870695 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:34.509177 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed5145b-4146-4e7b-a9c2-4cc68cc58506 admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:16:34.523738 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013737fb-fce7-453e-b348-567075a21a27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:16:34.525320 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013737fb-fce7-453e-b348-567075a21a27 admin admin] REQUEST_METHOD: `PUT` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:34.529302 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013737fb-fce7-453e-b348-567075a21a27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:34.530536 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013737fb-fce7-453e-b348-567075a21a27 admin admin] PATH_INFO: `/v3/projects/7b7f785828b94d3780612bcb8e955291/users/70fd3be71f974fe28c512174ce2d5293/roles/580aa7dbb38d4cff8f9d1bb8983f99ef` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:34.566075 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-154cdbf4-6631-4486-a547-3a3d62870695 None None] MFA Rules not processed for user `385444b8b35a4ad2a3617d6b9f24878a`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:16:34.567351 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013737fb-fce7-453e-b348-567075a21a27 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b7f785828b94d3780612bcb8e955291, user_id=70fd3be71f974fe28c512174ce2d5293, role_id=580aa7dbb38d4cff8f9d1bb8983f99ef)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:16:34.570304 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013737fb-fce7-453e-b348-567075a21a27 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:16:34.571859 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-154cdbf4-6631-4486-a547-3a3d62870695 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:16:34.678895 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 844/1689] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Aug 24 06:16:34 2022] PUT /identity/v3/projects/7b7f785828b94d3780612bcb8e955291/users/70fd3be71f974fe28c512174ce2d5293/roles/580aa7dbb38d4cff8f9d1bb8983f99ef => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:16:34.680318 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8b9ccb-e5d4-4904-b60b-f401f64cc327 None None] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:34.680726 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8b9ccb-e5d4-4904-b60b-f401f64cc327 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:34.681097 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8b9ccb-e5d4-4904-b60b-f401f64cc327 None None] PATH_INFO: `/` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:34.689886 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 845/1690] 10.222.0.62 () {54 vars in 961 bytes} [Wed Aug 24 06:16:34 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:16:34.693001 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918736ba-8506-4212-a331-5288cf52cc90 None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:34.702635 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918736ba-8506-4212-a331-5288cf52cc90 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:34.703086 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918736ba-8506-4212-a331-5288cf52cc90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:34.765987 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 846/1691] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:16:34 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:16:34.767399 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683008cb-22a8-4f9f-8dd2-eec786c27ceb None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:34.768206 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683008cb-22a8-4f9f-8dd2-eec786c27ceb None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:34.768511 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683008cb-22a8-4f9f-8dd2-eec786c27ceb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:34.775205 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-918736ba-8506-4212-a331-5288cf52cc90 None None] MFA Rules not processed for user `70fd3be71f974fe28c512174ce2d5293`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:16:34.776437 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-918736ba-8506-4212-a331-5288cf52cc90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:16:34.804521 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-683008cb-22a8-4f9f-8dd2-eec786c27ceb None None] MFA Rules not processed for user `abe9d8c9291845dfa4ef55034e5c9e32`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:16:34.806840 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-683008cb-22a8-4f9f-8dd2-eec786c27ceb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:16:34.921683 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 846/1692] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:16:34 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:16:34.925825 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918736ba-8506-4212-a331-5288cf52cc90 None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:16:34.930756 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-918736ba-8506-4212-a331-5288cf52cc90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:16:34.964252 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 847/1693] 10.222.0.62 () {58 vars in 1058 bytes} [Wed Aug 24 06:16:34 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:16:34.965163 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683008cb-22a8-4f9f-8dd2-eec786c27ceb None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:16:35.045115 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=94,keystone:INSERT=4,keystone:DELETE=2 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:16:35.090626 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683008cb-22a8-4f9f-8dd2-eec786c27ceb None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:16:35.095636 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55136bd9-74e9-45f7-8dc0-5630c39a14de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:16:35.096316 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55136bd9-74e9-45f7-8dc0-5630c39a14de admin admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:35.096632 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55136bd9-74e9-45f7-8dc0-5630c39a14de admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:35.096905 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55136bd9-74e9-45f7-8dc0-5630c39a14de admin admin] PATH_INFO: `/v3/projects` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:35.097844 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55136bd9-74e9-45f7-8dc0-5630c39a14de admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:16:35.100078 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918736ba-8506-4212-a331-5288cf52cc90 None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:16:35.100344 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55136bd9-74e9-45f7-8dc0-5630c39a14de admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:16:35.107817 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23248ace-824c-4e19-86d3-4d32dde1d44c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c4def55ac0e48b29836bc5c64bc7402', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:16:35.108824 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23248ace-824c-4e19-86d3-4d32dde1d44c service cinder] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:35.109125 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23248ace-824c-4e19-86d3-4d32dde1d44c service cinder] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:35.109388 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23248ace-824c-4e19-86d3-4d32dde1d44c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:35.190080 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 848/1694] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Aug 24 06:16:34 2022] POST /identity/v3/projects => generated 365 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:16:35.191972 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55136bd9-74e9-45f7-8dc0-5630c39a14de admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:16:35.223057 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23248ace-824c-4e19-86d3-4d32dde1d44c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:16:35.228215 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23248ace-824c-4e19-86d3-4d32dde1d44c service cinder] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:16:35.262821 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 847/1695] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:16:34 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:16:35.263986 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23248ace-824c-4e19-86d3-4d32dde1d44c service cinder] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:16:35.302994 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23248ace-824c-4e19-86d3-4d32dde1d44c service cinder] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:16:35.356457 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55136bd9-74e9-45f7-8dc0-5630c39a14de admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:16:35.370462 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c00ec5c-913c-4719-8f77-ea67dc413a76 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abe9d8c9291845dfa4ef55034e5c9e32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:16:35.371129 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c00ec5c-913c-4719-8f77-ea67dc413a76 service placement] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:35.371387 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c00ec5c-913c-4719-8f77-ea67dc413a76 service placement] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:35.371665 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c00ec5c-913c-4719-8f77-ea67dc413a76 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:35.374251 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:16:35.395138 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c00ec5c-913c-4719-8f77-ea67dc413a76 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:16:35.396648 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c00ec5c-913c-4719-8f77-ea67dc413a76 service placement] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:16:35.427849 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 849/1696] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:16:35 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:16:35.463433 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00265548-3a23-4b7b-b8e4-fa29147b25b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:16:35.464636 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00265548-3a23-4b7b-b8e4-fa29147b25b4 admin admin] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:35.465198 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00265548-3a23-4b7b-b8e4-fa29147b25b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:35.465773 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00265548-3a23-4b7b-b8e4-fa29147b25b4 admin admin] PATH_INFO: `/v3/users` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:35.468838 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00265548-3a23-4b7b-b8e4-fa29147b25b4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:16:35.472999 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00265548-3a23-4b7b-b8e4-fa29147b25b4 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:16:35.560105 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c00ec5c-913c-4719-8f77-ea67dc413a76 service placement] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:16:35.561824 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-1c00ec5c-913c-4719-8f77-ea67dc413a76 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:16:35.570718 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 848/1697] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Aug 24 06:16:35 2022] POST /identity/v3/users => generated 289 bytes in 307 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:16:35.575216 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00265548-3a23-4b7b-b8e4-fa29147b25b4 admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:16:35.591207 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00265548-3a23-4b7b-b8e4-fa29147b25b4 admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:16:35.605779 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6532a1-89a0-4adc-995c-93ba96d9a35b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:16:35.607995 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6532a1-89a0-4adc-995c-93ba96d9a35b admin admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:35.608585 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6532a1-89a0-4adc-995c-93ba96d9a35b admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:35.609051 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6532a1-89a0-4adc-995c-93ba96d9a35b admin admin] PATH_INFO: `/v3/roles` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:35.618058 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6532a1-89a0-4adc-995c-93ba96d9a35b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:16:35.620030 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6532a1-89a0-4adc-995c-93ba96d9a35b admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:16:35.645130 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 849/1698] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Aug 24 06:16:35 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 06:16:35.649537 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6532a1-89a0-4adc-995c-93ba96d9a35b admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:16:35.666181 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6532a1-89a0-4adc-995c-93ba96d9a35b admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:16:35.679906 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffd2931-2b5c-49b8-9a35-9c910d7cf0b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:16:35.681401 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffd2931-2b5c-49b8-9a35-9c910d7cf0b8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:35.681401 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffd2931-2b5c-49b8-9a35-9c910d7cf0b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:35.681401 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffd2931-2b5c-49b8-9a35-9c910d7cf0b8 admin admin] PATH_INFO: `/v3/projects/c94e7f91b064485c8356fe4181ac332c/users/78e123b5d96347548ec89d84f8b2acd0/roles/49e47cd157d743f6905f1db07ec878a7` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:35.693847 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c00ec5c-913c-4719-8f77-ea67dc413a76 service placement] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:16:35.698119 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c091a1fa-3f2b-4b55-9e5c-d1c9c20c16c9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c4def55ac0e48b29836bc5c64bc7402', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:16:35.698755 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c091a1fa-3f2b-4b55-9e5c-d1c9c20c16c9 service cinder] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:35.699038 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c091a1fa-3f2b-4b55-9e5c-d1c9c20c16c9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:35.699313 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c091a1fa-3f2b-4b55-9e5c-d1c9c20c16c9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:35.703671 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-c091a1fa-3f2b-4b55-9e5c-d1c9c20c16c9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:16:35.723458 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffd2931-2b5c-49b8-9a35-9c910d7cf0b8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c94e7f91b064485c8356fe4181ac332c, user_id=78e123b5d96347548ec89d84f8b2acd0, role_id=49e47cd157d743f6905f1db07ec878a7)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:16:35.726561 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffd2931-2b5c-49b8-9a35-9c910d7cf0b8 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:16:35.810322 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c091a1fa-3f2b-4b55-9e5c-d1c9c20c16c9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:16:35.811813 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c091a1fa-3f2b-4b55-9e5c-d1c9c20c16c9 service cinder] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:16:35.836113 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 850/1699] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:16:35 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 276 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 06:16:35.844311 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 850/1700] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Aug 24 06:16:35 2022] PUT /identity/v3/projects/c94e7f91b064485c8356fe4181ac332c/users/78e123b5d96347548ec89d84f8b2acd0/roles/49e47cd157d743f6905f1db07ec878a7 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:16:35.850875 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c091a1fa-3f2b-4b55-9e5c-d1c9c20c16c9 service cinder] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:16:35.870884 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c091a1fa-3f2b-4b55-9e5c-d1c9c20c16c9 service cinder] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:16:36.018106 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843595a9-f43a-4dc9-b497-fed242af2e5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:16:36.019066 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843595a9-f43a-4dc9-b497-fed242af2e5e admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:36.019346 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843595a9-f43a-4dc9-b497-fed242af2e5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:36.019618 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843595a9-f43a-4dc9-b497-fed242af2e5e admin admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:36.020617 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843595a9-f43a-4dc9-b497-fed242af2e5e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:16:36.022930 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843595a9-f43a-4dc9-b497-fed242af2e5e admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:16:36.057872 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 851/1701] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Aug 24 06:16:35 2022] GET /identity/v3/roles => generated 1420 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:16:36.061773 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffd2931-2b5c-49b8-9a35-9c910d7cf0b8 admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:16:36.077581 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffd2931-2b5c-49b8-9a35-9c910d7cf0b8 admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:16:36.091556 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd4b207-161b-40d1-9ac6-692ac7a96390 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:16:36.092301 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd4b207-161b-40d1-9ac6-692ac7a96390 admin admin] REQUEST_METHOD: `PUT` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:36.092718 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd4b207-161b-40d1-9ac6-692ac7a96390 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:36.093060 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd4b207-161b-40d1-9ac6-692ac7a96390 admin admin] PATH_INFO: `/v3/projects/c94e7f91b064485c8356fe4181ac332c/users/78e123b5d96347548ec89d84f8b2acd0/roles/49e47cd157d743f6905f1db07ec878a7` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:36.096951 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd4b207-161b-40d1-9ac6-692ac7a96390 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c94e7f91b064485c8356fe4181ac332c, user_id=78e123b5d96347548ec89d84f8b2acd0, role_id=49e47cd157d743f6905f1db07ec878a7)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:16:36.099964 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd4b207-161b-40d1-9ac6-692ac7a96390 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:16:36.139395 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775b005f-67aa-497c-b830-614be867652e None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:36.139523 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775b005f-67aa-497c-b830-614be867652e None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:36.139731 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775b005f-67aa-497c-b830-614be867652e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:36.150474 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 851/1702] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Aug 24 06:16:36 2022] PUT /identity/v3/projects/c94e7f91b064485c8356fe4181ac332c/users/78e123b5d96347548ec89d84f8b2acd0/roles/49e47cd157d743f6905f1db07ec878a7 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:16:36.197594 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-775b005f-67aa-497c-b830-614be867652e None None] MFA Rules not processed for user `78e123b5d96347548ec89d84f8b2acd0`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:16:36.202959 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-775b005f-67aa-497c-b830-614be867652e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:16:36.242828 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:16:36.415103 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=90,keystone:DELETE=5,keystone:INSERT=5 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:16:36.427963 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 852/1703] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:16:36 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 286 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:16:36.444375 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd4b207-161b-40d1-9ac6-692ac7a96390 admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:16:36.549015 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd4b207-161b-40d1-9ac6-692ac7a96390 admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:16:36.567046 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e0d06d-c6a6-433f-9ddc-520b1fcb6e32 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c4def55ac0e48b29836bc5c64bc7402', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:16:36.567690 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e0d06d-c6a6-433f-9ddc-520b1fcb6e32 service cinder] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:36.567954 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e0d06d-c6a6-433f-9ddc-520b1fcb6e32 service cinder] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:36.568222 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e0d06d-c6a6-433f-9ddc-520b1fcb6e32 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:36.584968 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e0d06d-c6a6-433f-9ddc-520b1fcb6e32 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:16:36.587373 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e0d06d-c6a6-433f-9ddc-520b1fcb6e32 service cinder] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:16:36.602208 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 852/1704] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:16:36 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:16:36.635341 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775b005f-67aa-497c-b830-614be867652e None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:16:36.636946 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-775b005f-67aa-497c-b830-614be867652e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:16:36.791970 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775b005f-67aa-497c-b830-614be867652e None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:16:36.856116 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702473e8-24c1-4419-8ce9-a3fe2c7196ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:16:36.860159 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702473e8-24c1-4419-8ce9-a3fe2c7196ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:36.860159 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702473e8-24c1-4419-8ce9-a3fe2c7196ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:36.860159 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702473e8-24c1-4419-8ce9-a3fe2c7196ec admin admin] PATH_INFO: `/v3/users/75db228aee9b4903aadebb69713cc36d` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:36.861204 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702473e8-24c1-4419-8ce9-a3fe2c7196ec admin admin] RBAC: Authorizing `identity:delete_user(user_id=75db228aee9b4903aadebb69713cc36d)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:16:36.863328 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702473e8-24c1-4419-8ce9-a3fe2c7196ec admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:16:37.107330 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-702473e8-24c1-4419-8ce9-a3fe2c7196ec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '75db228aee9b4903aadebb69713cc36d'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:16:37.139982 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-702473e8-24c1-4419-8ce9-a3fe2c7196ec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '75db228aee9b4903aadebb69713cc36d'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:16:37.141015 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-702473e8-24c1-4419-8ce9-a3fe2c7196ec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '75db228aee9b4903aadebb69713cc36d'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:16:37.236717 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-702473e8-24c1-4419-8ce9-a3fe2c7196ec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '75db228aee9b4903aadebb69713cc36d'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:16:37.271123 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-702473e8-24c1-4419-8ce9-a3fe2c7196ec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '75db228aee9b4903aadebb69713cc36d'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:16:37.289035 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 853/1705] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Aug 24 06:16:36 2022] DELETE /identity/v3/users/75db228aee9b4903aadebb69713cc36d => generated 0 bytes in 652 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:16:37.409584 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:16:38.377153 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:16:38.377631 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e0d06d-c6a6-433f-9ddc-520b1fcb6e32 service cinder] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:16:38.390816 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e0d06d-c6a6-433f-9ddc-520b1fcb6e32 service cinder] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:16:38.533671 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702473e8-24c1-4419-8ce9-a3fe2c7196ec admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:16:38.535184 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-702473e8-24c1-4419-8ce9-a3fe2c7196ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:16:38.569449 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ad4043-9074-434e-878e-dbab19521950 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:16:38.570821 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ad4043-9074-434e-878e-dbab19521950 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:38.571363 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ad4043-9074-434e-878e-dbab19521950 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:38.571927 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ad4043-9074-434e-878e-dbab19521950 admin admin] PATH_INFO: `/v3/users/70fd3be71f974fe28c512174ce2d5293` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:38.575490 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ad4043-9074-434e-878e-dbab19521950 admin admin] RBAC: Authorizing `identity:delete_user(user_id=70fd3be71f974fe28c512174ce2d5293)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:16:38.580241 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ad4043-9074-434e-878e-dbab19521950 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:16:38.767901 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702473e8-24c1-4419-8ce9-a3fe2c7196ec admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:16:38.781741 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2cb1a1-ae0c-4b12-9b68-de9e4b8ace87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:16:38.782889 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2cb1a1-ae0c-4b12-9b68-de9e4b8ace87 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:38.783346 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2cb1a1-ae0c-4b12-9b68-de9e4b8ace87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:38.783928 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2cb1a1-ae0c-4b12-9b68-de9e4b8ace87 admin admin] PATH_INFO: `/v3/projects/2527202c380e4491b7509f3b8a6fd61d` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:38.785960 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2cb1a1-ae0c-4b12-9b68-de9e4b8ace87 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2527202c380e4491b7509f3b8a6fd61d)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:16:38.788322 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2cb1a1-ae0c-4b12-9b68-de9e4b8ace87 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:16:38.898914 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-31ad4043-9074-434e-878e-dbab19521950 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '70fd3be71f974fe28c512174ce2d5293'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:16:38.915504 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-31ad4043-9074-434e-878e-dbab19521950 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '70fd3be71f974fe28c512174ce2d5293'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:16:38.916829 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-31ad4043-9074-434e-878e-dbab19521950 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '70fd3be71f974fe28c512174ce2d5293'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:16:38.920759 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-31ad4043-9074-434e-878e-dbab19521950 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '70fd3be71f974fe28c512174ce2d5293'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:16:39.007530 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-aa2cb1a1-ae0c-4b12-9b68-de9e4b8ace87 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2527202c380e4491b7509f3b8a6fd61d', 'request_id': 'req-aa2cb1a1-ae0c-4b12-9b68-de9e4b8ace87'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:16:39.016127 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-aa2cb1a1-ae0c-4b12-9b68-de9e4b8ace87 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2527202c380e4491b7509f3b8a6fd61d', 'request_id': 'req-aa2cb1a1-ae0c-4b12-9b68-de9e4b8ace87'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:16:39.027327 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-aa2cb1a1-ae0c-4b12-9b68-de9e4b8ace87 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2527202c380e4491b7509f3b8a6fd61d', 'request_id': 'req-aa2cb1a1-ae0c-4b12-9b68-de9e4b8ace87'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:16:39.030829 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-31ad4043-9074-434e-878e-dbab19521950 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '70fd3be71f974fe28c512174ce2d5293'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:16:39.057291 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 853/1706] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Aug 24 06:16:38 2022] DELETE /identity/v3/users/70fd3be71f974fe28c512174ce2d5293 => generated 0 bytes in 684 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:16:39.060813 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb398d39-a2bb-48a1-9a85-3ebeccb64066 None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:39.060813 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb398d39-a2bb-48a1-9a85-3ebeccb64066 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:39.060813 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb398d39-a2bb-48a1-9a85-3ebeccb64066 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:39.089780 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:16:39.105407 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-eb398d39-a2bb-48a1-9a85-3ebeccb64066 None None] MFA Rules not processed for user `5059723f417445fa934ecfe3e50d0197`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:16:39.106943 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-eb398d39-a2bb-48a1-9a85-3ebeccb64066 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:16:39.134544 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-aa2cb1a1-ae0c-4b12-9b68-de9e4b8ace87 admin admin] The token cache is being invalidate because project 2527202c380e4491b7509f3b8a6fd61d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81852) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:16:39.134900 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-aa2cb1a1-ae0c-4b12-9b68-de9e4b8ace87 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:16:39.136022 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-aa2cb1a1-ae0c-4b12-9b68-de9e4b8ace87 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:16:39.141806 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 854/1707] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Aug 24 06:16:38 2022] DELETE /identity/v3/projects/2527202c380e4491b7509f3b8a6fd61d => generated 0 bytes in 606 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 06:16:39.145519 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97596790-5cc0-4598-a27c-d429ad6c3c1b None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:39.146031 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97596790-5cc0-4598-a27c-d429ad6c3c1b None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:39.146365 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97596790-5cc0-4598-a27c-d429ad6c3c1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:39.234702 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 854/1708] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:16:39 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:16:39.234702 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb398d39-a2bb-48a1-9a85-3ebeccb64066 None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:16:39.234881 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-97596790-5cc0-4598-a27c-d429ad6c3c1b None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:16:39.234953 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-eb398d39-a2bb-48a1-9a85-3ebeccb64066 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:16:39.240262 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-97596790-5cc0-4598-a27c-d429ad6c3c1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:16:39.292114 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:16:39.390955 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb398d39-a2bb-48a1-9a85-3ebeccb64066 None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:16:39.392596 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 855/1709] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:16:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:16:39.397426 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97596790-5cc0-4598-a27c-d429ad6c3c1b None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:16:39.397426 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-97596790-5cc0-4598-a27c-d429ad6c3c1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:16:39.405713 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c747301-13e4-4ea4-ac01-1b44f3b0cf9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:16:39.405713 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c747301-13e4-4ea4-ac01-1b44f3b0cf9f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:39.406050 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c747301-13e4-4ea4-ac01-1b44f3b0cf9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:39.406347 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c747301-13e4-4ea4-ac01-1b44f3b0cf9f admin admin] PATH_INFO: `/v3/users/37c7468a5a3641999d6cb87d62394b3f` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:39.408593 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c747301-13e4-4ea4-ac01-1b44f3b0cf9f admin admin] RBAC: Authorizing `identity:delete_user(user_id=37c7468a5a3641999d6cb87d62394b3f)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:16:39.413572 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c747301-13e4-4ea4-ac01-1b44f3b0cf9f admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:16:39.594095 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97596790-5cc0-4598-a27c-d429ad6c3c1b None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:16:39.614213 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e107777d-21b5-4bfb-a273-343791982962 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb6d3743340241f98cacb9e0d063298c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:16:39.615070 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e107777d-21b5-4bfb-a273-343791982962 service nova] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:39.615351 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e107777d-21b5-4bfb-a273-343791982962 service nova] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:39.615990 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e107777d-21b5-4bfb-a273-343791982962 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:39.631658 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e107777d-21b5-4bfb-a273-343791982962 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:16:39.634622 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e107777d-21b5-4bfb-a273-343791982962 service nova] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:16:39.652278 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 856/1710] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:16:39 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 259 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:16:39.665570 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e107777d-21b5-4bfb-a273-343791982962 service nova] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:16:39.670708 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-e107777d-21b5-4bfb-a273-343791982962 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:16:39.736786 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-4c747301-13e4-4ea4-ac01-1b44f3b0cf9f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '37c7468a5a3641999d6cb87d62394b3f'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:16:39.764741 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-4c747301-13e4-4ea4-ac01-1b44f3b0cf9f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '37c7468a5a3641999d6cb87d62394b3f'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:16:39.765514 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-4c747301-13e4-4ea4-ac01-1b44f3b0cf9f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '37c7468a5a3641999d6cb87d62394b3f'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:16:39.766333 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-4c747301-13e4-4ea4-ac01-1b44f3b0cf9f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '37c7468a5a3641999d6cb87d62394b3f'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:16:39.823679 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e107777d-21b5-4bfb-a273-343791982962 service nova] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:16:39.834444 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68870575-55c9-4486-8564-1ed4960962bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bb6c79db75e4d169331c02133d1ec32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:16:39.835097 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68870575-55c9-4486-8564-1ed4960962bc service neutron] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:39.835381 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68870575-55c9-4486-8564-1ed4960962bc service neutron] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:39.835658 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68870575-55c9-4486-8564-1ed4960962bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:39.838159 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-68870575-55c9-4486-8564-1ed4960962bc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:16:39.866651 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68870575-55c9-4486-8564-1ed4960962bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:16:39.868149 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68870575-55c9-4486-8564-1ed4960962bc service neutron] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:16:39.905200 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 857/1711] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:16:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:16:39.911087 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-4c747301-13e4-4ea4-ac01-1b44f3b0cf9f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '37c7468a5a3641999d6cb87d62394b3f'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:16:39.994177 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 855/1712] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Aug 24 06:16:39 2022] DELETE /identity/v3/users/37c7468a5a3641999d6cb87d62394b3f => generated 0 bytes in 762 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:16:40.579107 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:16:40.744124 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68870575-55c9-4486-8564-1ed4960962bc service neutron] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:16:40.746711 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-68870575-55c9-4486-8564-1ed4960962bc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:16:40.788213 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68870575-55c9-4486-8564-1ed4960962bc service neutron] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:16:40.957685 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:16:40.964995 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bb32cb-0fb1-44bd-8f5d-18ca5b8ba933 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:16:40.965644 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bb32cb-0fb1-44bd-8f5d-18ca5b8ba933 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:40.966137 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bb32cb-0fb1-44bd-8f5d-18ca5b8ba933 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:40.966623 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bb32cb-0fb1-44bd-8f5d-18ca5b8ba933 admin admin] PATH_INFO: `/v3/projects/7b7f785828b94d3780612bcb8e955291` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:40.968661 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bb32cb-0fb1-44bd-8f5d-18ca5b8ba933 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7b7f785828b94d3780612bcb8e955291)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:16:40.973608 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bb32cb-0fb1-44bd-8f5d-18ca5b8ba933 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:16:41.082351 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c747301-13e4-4ea4-ac01-1b44f3b0cf9f admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:16:41.085074 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-4c747301-13e4-4ea4-ac01-1b44f3b0cf9f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:16:41.119996 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c747301-13e4-4ea4-ac01-1b44f3b0cf9f admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:16:41.126779 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34199086-d7df-4f6d-bcc4-5d0ae17e4cb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:16:41.127697 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34199086-d7df-4f6d-bcc4-5d0ae17e4cb7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:41.128947 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34199086-d7df-4f6d-bcc4-5d0ae17e4cb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:41.129436 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34199086-d7df-4f6d-bcc4-5d0ae17e4cb7 admin admin] PATH_INFO: `/v3/projects/889ae5dfed77476a834dade5174d751b` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:41.130645 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34199086-d7df-4f6d-bcc4-5d0ae17e4cb7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=889ae5dfed77476a834dade5174d751b)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:16:41.132393 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: WARNING py.warnings [None req-34199086-d7df-4f6d-bcc4-5d0ae17e4cb7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 06:16:41.132393 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: warnings.warn(msg) Aug 24 06:16:41.132393 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: Aug 24 06:16:41.133658 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34199086-d7df-4f6d-bcc4-5d0ae17e4cb7 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:16:41.150133 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:16:41.227065 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-c7bb32cb-0fb1-44bd-8f5d-18ca5b8ba933 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7b7f785828b94d3780612bcb8e955291', 'request_id': 'req-c7bb32cb-0fb1-44bd-8f5d-18ca5b8ba933'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:16:41.233210 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-c7bb32cb-0fb1-44bd-8f5d-18ca5b8ba933 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7b7f785828b94d3780612bcb8e955291', 'request_id': 'req-c7bb32cb-0fb1-44bd-8f5d-18ca5b8ba933'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:16:41.241472 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-c7bb32cb-0fb1-44bd-8f5d-18ca5b8ba933 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7b7f785828b94d3780612bcb8e955291', 'request_id': 'req-c7bb32cb-0fb1-44bd-8f5d-18ca5b8ba933'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:16:41.332730 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-c7bb32cb-0fb1-44bd-8f5d-18ca5b8ba933 admin admin] The token cache is being invalidate because project 7b7f785828b94d3780612bcb8e955291 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81852) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:16:41.333027 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-c7bb32cb-0fb1-44bd-8f5d-18ca5b8ba933 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:16:41.333729 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-c7bb32cb-0fb1-44bd-8f5d-18ca5b8ba933 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:16:41.335636 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 858/1713] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Aug 24 06:16:40 2022] DELETE /identity/v3/projects/7b7f785828b94d3780612bcb8e955291 => generated 0 bytes in 593 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:16:41.340336 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bb32cb-0fb1-44bd-8f5d-18ca5b8ba933 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:16:41.341773 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-c7bb32cb-0fb1-44bd-8f5d-18ca5b8ba933 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:16:41.343503 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-34199086-d7df-4f6d-bcc4-5d0ae17e4cb7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '889ae5dfed77476a834dade5174d751b', 'request_id': 'req-34199086-d7df-4f6d-bcc4-5d0ae17e4cb7'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:16:41.352879 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-34199086-d7df-4f6d-bcc4-5d0ae17e4cb7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '889ae5dfed77476a834dade5174d751b', 'request_id': 'req-34199086-d7df-4f6d-bcc4-5d0ae17e4cb7'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:16:41.370209 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-34199086-d7df-4f6d-bcc4-5d0ae17e4cb7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '889ae5dfed77476a834dade5174d751b', 'request_id': 'req-34199086-d7df-4f6d-bcc4-5d0ae17e4cb7'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:16:41.467956 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-34199086-d7df-4f6d-bcc4-5d0ae17e4cb7 admin admin] The token cache is being invalidate because project 889ae5dfed77476a834dade5174d751b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81851) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:16:41.468377 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-34199086-d7df-4f6d-bcc4-5d0ae17e4cb7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:16:41.469297 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-34199086-d7df-4f6d-bcc4-5d0ae17e4cb7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:16:41.472296 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 856/1714] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Aug 24 06:16:41 2022] DELETE /identity/v3/projects/889ae5dfed77476a834dade5174d751b => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 06:16:41.478414 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c636bd1-b804-470e-99f3-a5f1463fb48d None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:41.480234 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c636bd1-b804-470e-99f3-a5f1463fb48d None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:41.480723 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c636bd1-b804-470e-99f3-a5f1463fb48d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:41.481624 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bb32cb-0fb1-44bd-8f5d-18ca5b8ba933 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:16:41.517239 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-2c636bd1-b804-470e-99f3-a5f1463fb48d None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:16:41.520057 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-2c636bd1-b804-470e-99f3-a5f1463fb48d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:16:41.616302 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69398090-b927-49ca-8c59-4c97a4be528b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:16:41.617826 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69398090-b927-49ca-8c59-4c97a4be528b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:41.618817 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69398090-b927-49ca-8c59-4c97a4be528b admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:41.619237 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69398090-b927-49ca-8c59-4c97a4be528b admin admin] PATH_INFO: `/v3/users/78e123b5d96347548ec89d84f8b2acd0` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:41.621444 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69398090-b927-49ca-8c59-4c97a4be528b admin admin] RBAC: Authorizing `identity:delete_user(user_id=78e123b5d96347548ec89d84f8b2acd0)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:16:41.624844 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69398090-b927-49ca-8c59-4c97a4be528b admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:16:41.661270 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 857/1715] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:16:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:16:41.662774 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee10142e-853b-4ba5-85f9-de34e8c28ff3 None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:41.663651 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee10142e-853b-4ba5-85f9-de34e8c28ff3 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:41.664082 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee10142e-853b-4ba5-85f9-de34e8c28ff3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:41.727360 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-ee10142e-853b-4ba5-85f9-de34e8c28ff3 None None] MFA Rules not processed for user `6abfd0a28f9e482badae7018f1b7f9a1`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:16:41.728626 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-ee10142e-853b-4ba5-85f9-de34e8c28ff3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:16:41.840658 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 858/1716] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:16:41 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:16:41.842379 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265c59ed-94fb-4b49-a928-efc3d54b17e4 None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:41.842672 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265c59ed-94fb-4b49-a928-efc3d54b17e4 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:41.843070 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265c59ed-94fb-4b49-a928-efc3d54b17e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:41.938523 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-265c59ed-94fb-4b49-a928-efc3d54b17e4 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:16:41.940074 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-265c59ed-94fb-4b49-a928-efc3d54b17e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:16:41.952570 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-69398090-b927-49ca-8c59-4c97a4be528b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '78e123b5d96347548ec89d84f8b2acd0'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:16:41.963769 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-69398090-b927-49ca-8c59-4c97a4be528b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '78e123b5d96347548ec89d84f8b2acd0'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:16:41.966332 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-69398090-b927-49ca-8c59-4c97a4be528b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '78e123b5d96347548ec89d84f8b2acd0'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:16:41.988741 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 859/1717] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:16:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:16:41.990510 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada7f608-bd28-43c9-9be1-686bc88034b0 None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:41.990510 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada7f608-bd28-43c9-9be1-686bc88034b0 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:41.994794 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada7f608-bd28-43c9-9be1-686bc88034b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:42.076044 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-ada7f608-bd28-43c9-9be1-686bc88034b0 None None] MFA Rules not processed for user `6021a225b5fd4377a9f8120a84232709`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:16:42.077546 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-ada7f608-bd28-43c9-9be1-686bc88034b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:16:42.083134 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-69398090-b927-49ca-8c59-4c97a4be528b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '78e123b5d96347548ec89d84f8b2acd0'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:16:42.103371 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-69398090-b927-49ca-8c59-4c97a4be528b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '78e123b5d96347548ec89d84f8b2acd0'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:16:42.115159 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 859/1718] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Aug 24 06:16:41 2022] DELETE /identity/v3/users/78e123b5d96347548ec89d84f8b2acd0 => generated 0 bytes in 775 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:16:42.125783 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2817c55-0151-4afb-b588-3bc745f425ae None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:42.126995 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2817c55-0151-4afb-b588-3bc745f425ae None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:42.126995 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2817c55-0151-4afb-b588-3bc745f425ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:42.210609 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-f2817c55-0151-4afb-b588-3bc745f425ae None None] MFA Rules not processed for user `25908fb02f7f46f5b9e483811302a67e`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:16:42.216847 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-f2817c55-0151-4afb-b588-3bc745f425ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:16:42.245229 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 860/1719] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:16:41 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:16:42.246355 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada7f608-bd28-43c9-9be1-686bc88034b0 None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:16:42.257272 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-ada7f608-bd28-43c9-9be1-686bc88034b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:16:42.356550 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 860/1720] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:16:42 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:16:42.357634 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2817c55-0151-4afb-b588-3bc745f425ae None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:16:42.359620 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-f2817c55-0151-4afb-b588-3bc745f425ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:16:42.410346 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada7f608-bd28-43c9-9be1-686bc88034b0 None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:16:42.455688 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2817c55-0151-4afb-b588-3bc745f425ae None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:16:42.465278 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5e8bab-f414-478f-9144-62aafcee1f3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:16:42.468267 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5e8bab-f414-478f-9144-62aafcee1f3c admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:42.468804 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5e8bab-f414-478f-9144-62aafcee1f3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:42.469148 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5e8bab-f414-478f-9144-62aafcee1f3c admin admin] PATH_INFO: `/v3/domains` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:42.469931 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5e8bab-f414-478f-9144-62aafcee1f3c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:16:42.470299 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5e8bab-f414-478f-9144-62aafcee1f3c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:16:42.472214 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5e8bab-f414-478f-9144-62aafcee1f3c admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:16:42.485803 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 861/1721] 10.222.0.62 () {58 vars in 1153 bytes} [Wed Aug 24 06:16:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:16:42.487435 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d838f64b-1c2c-4f76-b59f-f4a8cc92a38c None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:42.487824 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d838f64b-1c2c-4f76-b59f-f4a8cc92a38c None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:42.488196 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d838f64b-1c2c-4f76-b59f-f4a8cc92a38c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:42.513126 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-d838f64b-1c2c-4f76-b59f-f4a8cc92a38c None None] MFA Rules not processed for user `6021a225b5fd4377a9f8120a84232709`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:16:42.516450 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-d838f64b-1c2c-4f76-b59f-f4a8cc92a38c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:16:42.567818 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afd90fb-8e52-420c-8668-c6a9dff9574b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb6d3743340241f98cacb9e0d063298c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:16:42.570523 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afd90fb-8e52-420c-8668-c6a9dff9574b service nova] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:42.570883 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afd90fb-8e52-420c-8668-c6a9dff9574b service nova] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:42.571152 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afd90fb-8e52-420c-8668-c6a9dff9574b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:42.576888 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-2afd90fb-8e52-420c-8668-c6a9dff9574b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:16:42.634615 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 862/1722] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:16:42 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:16:42.635461 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d838f64b-1c2c-4f76-b59f-f4a8cc92a38c None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:16:42.665174 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d838f64b-1c2c-4f76-b59f-f4a8cc92a38c None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:16:42.689023 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9b4575-cace-4d79-9dee-ff8dd51f780b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:16:42.689664 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9b4575-cace-4d79-9dee-ff8dd51f780b admin admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:42.689943 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9b4575-cace-4d79-9dee-ff8dd51f780b admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:42.691260 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9b4575-cace-4d79-9dee-ff8dd51f780b admin admin] PATH_INFO: `/v3/projects` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:42.691886 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9b4575-cace-4d79-9dee-ff8dd51f780b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:16:42.702182 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9b4575-cace-4d79-9dee-ff8dd51f780b admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:16:42.773488 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afd90fb-8e52-420c-8668-c6a9dff9574b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:16:42.774700 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afd90fb-8e52-420c-8668-c6a9dff9574b service nova] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:16:42.785217 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 861/1723] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:16:42 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 539 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:16:42.874724 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 863/1724] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Aug 24 06:16:42 2022] POST /identity/v3/projects => generated 387 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:16:42.879299 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afd90fb-8e52-420c-8668-c6a9dff9574b service nova] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:16:42.898016 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afd90fb-8e52-420c-8668-c6a9dff9574b service nova] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:16:42.911388 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:16:42.965735 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9b4575-cace-4d79-9dee-ff8dd51f780b admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:16:43.063094 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa29e421-7873-462b-8650-c14cfdab0776 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:16:43.064964 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa29e421-7873-462b-8650-c14cfdab0776 admin admin] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:43.065249 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa29e421-7873-462b-8650-c14cfdab0776 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:43.066740 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa29e421-7873-462b-8650-c14cfdab0776 admin admin] PATH_INFO: `/v3/users` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:43.068466 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa29e421-7873-462b-8650-c14cfdab0776 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:16:43.072952 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa29e421-7873-462b-8650-c14cfdab0776 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:16:43.109583 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9b4575-cace-4d79-9dee-ff8dd51f780b admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:16:43.117744 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4e821e-ab77-48c4-892f-5253cb321efe service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb6d3743340241f98cacb9e0d063298c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:16:43.118566 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4e821e-ab77-48c4-892f-5253cb321efe service nova] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:43.118944 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4e821e-ab77-48c4-892f-5253cb321efe service nova] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:43.119316 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4e821e-ab77-48c4-892f-5253cb321efe service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:43.184993 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 862/1725] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Aug 24 06:16:42 2022] POST /identity/v3/users => generated 294 bytes in 307 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:16:43.186127 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa29e421-7873-462b-8650-c14cfdab0776 admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:16:43.189267 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-fa29e421-7873-462b-8650-c14cfdab0776 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:16:43.228126 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa29e421-7873-462b-8650-c14cfdab0776 admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:16:43.229359 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4e821e-ab77-48c4-892f-5253cb321efe service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:16:43.233308 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4e821e-ab77-48c4-892f-5253cb321efe service nova] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:16:43.240829 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3881f2e8-8422-43ee-bfdc-789e0e32ebee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:16:43.241481 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3881f2e8-8422-43ee-bfdc-789e0e32ebee admin admin] REQUEST_METHOD: `DELETE` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:43.241763 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3881f2e8-8422-43ee-bfdc-789e0e32ebee admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:43.242036 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3881f2e8-8422-43ee-bfdc-789e0e32ebee admin admin] PATH_INFO: `/v3/users/114a88fb0dfe4f33abf5a654dee0df1e` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:43.246841 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3881f2e8-8422-43ee-bfdc-789e0e32ebee admin admin] RBAC: Authorizing `identity:delete_user(user_id=114a88fb0dfe4f33abf5a654dee0df1e)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:16:43.252178 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 864/1726] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:16:42 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 287 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 06:16:43.252633 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3881f2e8-8422-43ee-bfdc-789e0e32ebee admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:16:43.254613 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4e821e-ab77-48c4-892f-5253cb321efe service nova] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:16:43.255213 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-6e4e821e-ab77-48c4-892f-5253cb321efe service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:16:43.271652 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4e821e-ab77-48c4-892f-5253cb321efe service nova] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:16:43.273823 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:16:43.278623 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b11bf96-fa73-4085-a6ff-8267a1a49741 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:16:43.279403 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b11bf96-fa73-4085-a6ff-8267a1a49741 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:43.279763 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b11bf96-fa73-4085-a6ff-8267a1a49741 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:43.280105 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b11bf96-fa73-4085-a6ff-8267a1a49741 admin admin] PATH_INFO: `/v3/projects/c94e7f91b064485c8356fe4181ac332c` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:43.285640 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b11bf96-fa73-4085-a6ff-8267a1a49741 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c94e7f91b064485c8356fe4181ac332c)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:16:43.287855 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b11bf96-fa73-4085-a6ff-8267a1a49741 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:16:43.501615 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-3b11bf96-fa73-4085-a6ff-8267a1a49741 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c94e7f91b064485c8356fe4181ac332c', 'request_id': 'req-3b11bf96-fa73-4085-a6ff-8267a1a49741'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:16:43.511832 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-3881f2e8-8422-43ee-bfdc-789e0e32ebee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '114a88fb0dfe4f33abf5a654dee0df1e'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:16:43.512035 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-3b11bf96-fa73-4085-a6ff-8267a1a49741 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c94e7f91b064485c8356fe4181ac332c', 'request_id': 'req-3b11bf96-fa73-4085-a6ff-8267a1a49741'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:16:43.519916 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-3b11bf96-fa73-4085-a6ff-8267a1a49741 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c94e7f91b064485c8356fe4181ac332c', 'request_id': 'req-3b11bf96-fa73-4085-a6ff-8267a1a49741'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:16:43.543324 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-3881f2e8-8422-43ee-bfdc-789e0e32ebee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '114a88fb0dfe4f33abf5a654dee0df1e'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:16:43.545641 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-3881f2e8-8422-43ee-bfdc-789e0e32ebee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '114a88fb0dfe4f33abf5a654dee0df1e'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:16:43.546659 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-3881f2e8-8422-43ee-bfdc-789e0e32ebee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '114a88fb0dfe4f33abf5a654dee0df1e'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:16:43.625988 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-3b11bf96-fa73-4085-a6ff-8267a1a49741 admin admin] The token cache is being invalidate because project c94e7f91b064485c8356fe4181ac332c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81852) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:16:43.626281 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-3b11bf96-fa73-4085-a6ff-8267a1a49741 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:16:43.626962 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-3b11bf96-fa73-4085-a6ff-8267a1a49741 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:16:43.628951 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 865/1727] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Aug 24 06:16:43 2022] DELETE /identity/v3/projects/c94e7f91b064485c8356fe4181ac332c => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:16:43.630381 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b11bf96-fa73-4085-a6ff-8267a1a49741 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:16:43.633720 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-3b11bf96-fa73-4085-a6ff-8267a1a49741 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:16:43.705368 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-3881f2e8-8422-43ee-bfdc-789e0e32ebee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '114a88fb0dfe4f33abf5a654dee0df1e'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:16:43.738479 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 863/1728] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Aug 24 06:16:43 2022] DELETE /identity/v3/users/114a88fb0dfe4f33abf5a654dee0df1e => generated 0 bytes in 549 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:16:43.740635 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095d0221-7f62-48c2-b347-645843d25c8b None None] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:43.741016 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095d0221-7f62-48c2-b347-645843d25c8b None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:43.743174 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095d0221-7f62-48c2-b347-645843d25c8b None None] PATH_INFO: `/` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:43.744970 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 864/1729] 10.222.0.62 () {54 vars in 912 bytes} [Wed Aug 24 06:16:43 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:16:43.755926 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f16a81-72d1-44e6-8a12-89c5cdfdd736 None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:43.756587 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f16a81-72d1-44e6-8a12-89c5cdfdd736 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:43.757064 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f16a81-72d1-44e6-8a12-89c5cdfdd736 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:43.771773 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b11bf96-fa73-4085-a6ff-8267a1a49741 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:16:43.811819 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-94f16a81-72d1-44e6-8a12-89c5cdfdd736 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:16:43.813242 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-94f16a81-72d1-44e6-8a12-89c5cdfdd736 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:16:43.899724 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9189cc97-59bd-4a20-bfbf-f2ff56e5f006 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:16:43.900803 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9189cc97-59bd-4a20-bfbf-f2ff56e5f006 admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:43.901255 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9189cc97-59bd-4a20-bfbf-f2ff56e5f006 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:43.901696 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9189cc97-59bd-4a20-bfbf-f2ff56e5f006 admin admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:43.902693 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9189cc97-59bd-4a20-bfbf-f2ff56e5f006 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:16:43.906318 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9189cc97-59bd-4a20-bfbf-f2ff56e5f006 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:16:43.925617 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 866/1730] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Aug 24 06:16:43 2022] GET /identity/v3/roles => generated 1420 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:16:43.928680 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ef2e4b-7326-40ad-8bab-e72cb293e667 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:43.929207 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ef2e4b-7326-40ad-8bab-e72cb293e667 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:43.930276 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ef2e4b-7326-40ad-8bab-e72cb293e667 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:43.961157 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 865/1731] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:16:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:16:43.964072 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-86ef2e4b-7326-40ad-8bab-e72cb293e667 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:16:43.965822 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-86ef2e4b-7326-40ad-8bab-e72cb293e667 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:16:43.971422 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11313a2c-24dd-4795-b4d5-c659517a6e9c None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:43.971743 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11313a2c-24dd-4795-b4d5-c659517a6e9c None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:43.972024 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11313a2c-24dd-4795-b4d5-c659517a6e9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:43.982427 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 867/1732] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:16:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:16:43.985569 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ef2e4b-7326-40ad-8bab-e72cb293e667 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:16:44.002732 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ef2e4b-7326-40ad-8bab-e72cb293e667 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:16:44.010868 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491ef9c0-4da5-433c-86ac-f2fe2e1b05bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:16:44.011892 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491ef9c0-4da5-433c-86ac-f2fe2e1b05bc admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:44.012578 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491ef9c0-4da5-433c-86ac-f2fe2e1b05bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:44.013039 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491ef9c0-4da5-433c-86ac-f2fe2e1b05bc admin admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:44.016580 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491ef9c0-4da5-433c-86ac-f2fe2e1b05bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:16:44.017916 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491ef9c0-4da5-433c-86ac-f2fe2e1b05bc admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:16:44.032511 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 868/1733] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Aug 24 06:16:43 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:16:44.034340 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba7c9e9-cdcc-4d5f-9be7-c07cc1134e1a None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:44.035011 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba7c9e9-cdcc-4d5f-9be7-c07cc1134e1a None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:44.035487 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba7c9e9-cdcc-4d5f-9be7-c07cc1134e1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:44.064907 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-eba7c9e9-cdcc-4d5f-9be7-c07cc1134e1a None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:16:44.067766 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-eba7c9e9-cdcc-4d5f-9be7-c07cc1134e1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:16:44.080280 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 869/1734] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:16:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:16:44.082286 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba7c9e9-cdcc-4d5f-9be7-c07cc1134e1a None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:16:44.084131 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-eba7c9e9-cdcc-4d5f-9be7-c07cc1134e1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:16:44.093097 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-11313a2c-24dd-4795-b4d5-c659517a6e9c None None] MFA Rules not processed for user `0bb6c79db75e4d169331c02133d1ec32`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:16:44.098013 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-11313a2c-24dd-4795-b4d5-c659517a6e9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:16:44.196362 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba7c9e9-cdcc-4d5f-9be7-c07cc1134e1a None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:16:44.196798 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 866/1735] 10.222.0.62 () {58 vars in 1009 bytes} [Wed Aug 24 06:16:43 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:16:44.197951 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11313a2c-24dd-4795-b4d5-c659517a6e9c None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:16:44.202991 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56b1ebf-cf33-486c-bc83-642da39a670a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bb6c79db75e4d169331c02133d1ec32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:16:44.205956 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56b1ebf-cf33-486c-bc83-642da39a670a service neutron] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:44.205956 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56b1ebf-cf33-486c-bc83-642da39a670a service neutron] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:44.205956 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56b1ebf-cf33-486c-bc83-642da39a670a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:44.212455 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11313a2c-24dd-4795-b4d5-c659517a6e9c None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:16:44.216806 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56b1ebf-cf33-486c-bc83-642da39a670a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:16:44.218284 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64119a91-0241-4199-9940-bfaa2fe0cf9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:16:44.218966 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64119a91-0241-4199-9940-bfaa2fe0cf9e admin admin] REQUEST_METHOD: `PUT` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:44.219073 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56b1ebf-cf33-486c-bc83-642da39a670a service neutron] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:16:44.219234 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64119a91-0241-4199-9940-bfaa2fe0cf9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:44.219509 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64119a91-0241-4199-9940-bfaa2fe0cf9e admin admin] PATH_INFO: `/v3/projects/3b7c95ff81674cbebc48223c0fc09516/users/1efc05afaa684f79a5fd8ed9f4d53ff8/roles/580aa7dbb38d4cff8f9d1bb8983f99ef` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:44.234645 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 870/1736] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:16:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:16:44.237130 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455b1ad0-e980-42f9-b2f9-392ea445230a None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:44.237682 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455b1ad0-e980-42f9-b2f9-392ea445230a None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:44.238162 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455b1ad0-e980-42f9-b2f9-392ea445230a None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:44.240275 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 871/1737] 10.222.0.62 () {54 vars in 912 bytes} [Wed Aug 24 06:16:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:16:44.241398 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455b1ad0-e980-42f9-b2f9-392ea445230a None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:16:44.261888 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64119a91-0241-4199-9940-bfaa2fe0cf9e admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b7c95ff81674cbebc48223c0fc09516, user_id=1efc05afaa684f79a5fd8ed9f4d53ff8, role_id=580aa7dbb38d4cff8f9d1bb8983f99ef)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:16:44.268118 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64119a91-0241-4199-9940-bfaa2fe0cf9e admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:16:44.272077 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455b1ad0-e980-42f9-b2f9-392ea445230a None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:16:44.282796 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c4f69e-8924-44b9-962e-78d40a9f4509 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:16:44.285575 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c4f69e-8924-44b9-962e-78d40a9f4509 admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:44.286037 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c4f69e-8924-44b9-962e-78d40a9f4509 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:44.286496 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c4f69e-8924-44b9-962e-78d40a9f4509 admin admin] PATH_INFO: `/v3/domains` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:44.287379 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c4f69e-8924-44b9-962e-78d40a9f4509 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:16:44.287808 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c4f69e-8924-44b9-962e-78d40a9f4509 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:16:44.289918 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c4f69e-8924-44b9-962e-78d40a9f4509 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:16:44.308022 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 872/1738] 10.222.0.62 () {58 vars in 1153 bytes} [Wed Aug 24 06:16:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:16:44.309274 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c4f69e-8924-44b9-962e-78d40a9f4509 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:16:44.312717 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-b5c4f69e-8924-44b9-962e-78d40a9f4509 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:16:44.332581 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c4f69e-8924-44b9-962e-78d40a9f4509 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:16:44.336662 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdaca76-5526-4eae-ab76-22418a5512d7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bb6c79db75e4d169331c02133d1ec32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:16:44.337606 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdaca76-5526-4eae-ab76-22418a5512d7 service neutron] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:44.338108 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdaca76-5526-4eae-ab76-22418a5512d7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:44.338473 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdaca76-5526-4eae-ab76-22418a5512d7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:44.354982 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 867/1739] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Aug 24 06:16:44 2022] PUT /identity/v3/projects/3b7c95ff81674cbebc48223c0fc09516/users/1efc05afaa684f79a5fd8ed9f4d53ff8/roles/580aa7dbb38d4cff8f9d1bb8983f99ef => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:16:44.356755 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3549e658-c934-4558-ab17-8033c49e0826 None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:44.357143 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3549e658-c934-4558-ab17-8033c49e0826 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:44.357522 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3549e658-c934-4558-ab17-8033c49e0826 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:44.366473 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfdaca76-5526-4eae-ab76-22418a5512d7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:16:44.369486 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfdaca76-5526-4eae-ab76-22418a5512d7 service neutron] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:16:44.398648 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 873/1740] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:16:44 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:16:44.400309 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdaca76-5526-4eae-ab76-22418a5512d7 service neutron] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:16:44.409632 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-3549e658-c934-4558-ab17-8033c49e0826 None None] MFA Rules not processed for user `0bb6c79db75e4d169331c02133d1ec32`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:16:44.415328 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-3549e658-c934-4558-ab17-8033c49e0826 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:16:44.430369 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdaca76-5526-4eae-ab76-22418a5512d7 service neutron] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:16:44.447260 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:16:44.557710 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 868/1741] 10.222.0.62 () {58 vars in 1009 bytes} [Wed Aug 24 06:16:44 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:16:44.559080 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dbe582-2363-4b2f-bde2-8803ab169c39 None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:44.559474 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dbe582-2363-4b2f-bde2-8803ab169c39 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:44.559852 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dbe582-2363-4b2f-bde2-8803ab169c39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:44.584333 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d7a218-a829-4c64-9652-4cadaf3eb635 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:16:44.585683 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d7a218-a829-4c64-9652-4cadaf3eb635 admin admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:44.585965 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d7a218-a829-4c64-9652-4cadaf3eb635 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:44.587197 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d7a218-a829-4c64-9652-4cadaf3eb635 admin admin] PATH_INFO: `/v3/projects` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:44.588076 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d7a218-a829-4c64-9652-4cadaf3eb635 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:16:44.597626 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d7a218-a829-4c64-9652-4cadaf3eb635 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:16:44.600611 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-a9dbe582-2363-4b2f-bde2-8803ab169c39 None None] MFA Rules not processed for user `1efc05afaa684f79a5fd8ed9f4d53ff8`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:16:44.601993 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-a9dbe582-2363-4b2f-bde2-8803ab169c39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:16:44.716023 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 869/1742] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:16:44 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:16:44.716023 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9dbe582-2363-4b2f-bde2-8803ab169c39 None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:16:44.729887 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9dbe582-2363-4b2f-bde2-8803ab169c39 None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:16:44.734197 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9776f80-417e-40ce-a8e3-597dead7fc37 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bb6c79db75e4d169331c02133d1ec32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:16:44.735022 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9776f80-417e-40ce-a8e3-597dead7fc37 service neutron] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:44.735825 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9776f80-417e-40ce-a8e3-597dead7fc37 service neutron] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:44.735825 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9776f80-417e-40ce-a8e3-597dead7fc37 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:44.744278 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9776f80-417e-40ce-a8e3-597dead7fc37 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:16:44.746066 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9776f80-417e-40ce-a8e3-597dead7fc37 service neutron] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:16:44.763915 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 870/1743] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:16:44 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:16:44.764810 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9776f80-417e-40ce-a8e3-597dead7fc37 service neutron] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:16:44.766966 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 874/1744] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Aug 24 06:16:44 2022] POST /identity/v3/projects => generated 369 bytes in 368 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:16:44.771313 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d7a218-a829-4c64-9652-4cadaf3eb635 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:16:44.791886 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9776f80-417e-40ce-a8e3-597dead7fc37 service neutron] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:16:44.890387 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:16:44.908378 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d7a218-a829-4c64-9652-4cadaf3eb635 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:16:44.924268 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561ea287-b6d3-406b-87a8-5b340bf9f957 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:16:44.925332 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561ea287-b6d3-406b-87a8-5b340bf9f957 admin admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:44.926055 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561ea287-b6d3-406b-87a8-5b340bf9f957 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:44.926556 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561ea287-b6d3-406b-87a8-5b340bf9f957 admin admin] PATH_INFO: `/v3/users` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:44.927522 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561ea287-b6d3-406b-87a8-5b340bf9f957 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:16:44.928742 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c016d4a-da30-4c6e-839b-46feb3bac323 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:16:44.929431 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c016d4a-da30-4c6e-839b-46feb3bac323 admin admin] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:44.929709 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c016d4a-da30-4c6e-839b-46feb3bac323 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:44.929981 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c016d4a-da30-4c6e-839b-46feb3bac323 admin admin] PATH_INFO: `/v3/projects` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:44.930609 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c016d4a-da30-4c6e-839b-46feb3bac323 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:16:44.937457 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561ea287-b6d3-406b-87a8-5b340bf9f957 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:16:44.938318 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c016d4a-da30-4c6e-839b-46feb3bac323 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:16:45.082068 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 875/1745] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Aug 24 06:16:44 2022] POST /identity/v3/users => generated 285 bytes in 302 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 06:16:45.082263 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 871/1746] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Aug 24 06:16:44 2022] POST /identity/v3/projects => generated 387 bytes in 310 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:16:45.082342 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561ea287-b6d3-406b-87a8-5b340bf9f957 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:16:45.082433 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c016d4a-da30-4c6e-839b-46feb3bac323 admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:16:45.113227 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561ea287-b6d3-406b-87a8-5b340bf9f957 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:16:45.290817 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c016d4a-da30-4c6e-839b-46feb3bac323 admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:16:45.305486 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa35bda0-88b4-4d34-b812-abe53e2b92e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:16:45.306179 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa35bda0-88b4-4d34-b812-abe53e2b92e9 admin admin] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:45.306467 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa35bda0-88b4-4d34-b812-abe53e2b92e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:45.306739 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa35bda0-88b4-4d34-b812-abe53e2b92e9 admin admin] PATH_INFO: `/v3/users` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:45.308555 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa35bda0-88b4-4d34-b812-abe53e2b92e9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:16:45.311767 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a864627-c255-4a0e-bdb4-5361fa76df76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:16:45.312632 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a864627-c255-4a0e-bdb4-5361fa76df76 admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:45.313013 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa35bda0-88b4-4d34-b812-abe53e2b92e9 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:16:45.313216 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a864627-c255-4a0e-bdb4-5361fa76df76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:45.313630 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a864627-c255-4a0e-bdb4-5361fa76df76 admin admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:45.314454 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a864627-c255-4a0e-bdb4-5361fa76df76 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:16:45.316777 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a864627-c255-4a0e-bdb4-5361fa76df76 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:16:45.320570 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:16:45.353017 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 876/1747] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Aug 24 06:16:45 2022] GET /identity/v3/roles => generated 1420 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:16:45.354461 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a864627-c255-4a0e-bdb4-5361fa76df76 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:16:45.369689 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a864627-c255-4a0e-bdb4-5361fa76df76 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:16:45.375253 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a113144c-a2cb-4a54-bc1a-fa2f12bc450c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:16:45.376046 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a113144c-a2cb-4a54-bc1a-fa2f12bc450c admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:45.376472 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a113144c-a2cb-4a54-bc1a-fa2f12bc450c admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:45.376970 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a113144c-a2cb-4a54-bc1a-fa2f12bc450c admin admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:45.378314 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a113144c-a2cb-4a54-bc1a-fa2f12bc450c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:16:45.380154 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a113144c-a2cb-4a54-bc1a-fa2f12bc450c admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:16:45.404809 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 872/1748] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Aug 24 06:16:45 2022] POST /identity/v3/users => generated 300 bytes in 323 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:16:45.404809 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa35bda0-88b4-4d34-b812-abe53e2b92e9 admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:16:45.406374 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 877/1749] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Aug 24 06:16:45 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:16:45.406960 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-fa35bda0-88b4-4d34-b812-abe53e2b92e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:16:45.408017 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a113144c-a2cb-4a54-bc1a-fa2f12bc450c admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:16:45.438149 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a113144c-a2cb-4a54-bc1a-fa2f12bc450c admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:16:45.444055 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb29cb5-34d8-4012-9412-c49f9c0e70e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:16:45.444703 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb29cb5-34d8-4012-9412-c49f9c0e70e7 admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:45.444987 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb29cb5-34d8-4012-9412-c49f9c0e70e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:45.445748 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb29cb5-34d8-4012-9412-c49f9c0e70e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:45.446569 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb29cb5-34d8-4012-9412-c49f9c0e70e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:16:45.448551 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb29cb5-34d8-4012-9412-c49f9c0e70e7 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:16:45.461272 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 878/1750] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Aug 24 06:16:45 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:16:45.461843 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb29cb5-34d8-4012-9412-c49f9c0e70e7 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:16:45.476049 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb29cb5-34d8-4012-9412-c49f9c0e70e7 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:16:45.482926 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a037c0-5091-4ead-8cb4-442ab955ca60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:16:45.483918 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a037c0-5091-4ead-8cb4-442ab955ca60 admin admin] REQUEST_METHOD: `PUT` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:45.490377 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a037c0-5091-4ead-8cb4-442ab955ca60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:45.490377 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a037c0-5091-4ead-8cb4-442ab955ca60 admin admin] PATH_INFO: `/v3/projects/983f56f36a63477690fe70282ae57787/users/e632e4d5279c4046b2c0976449152df4/roles/580aa7dbb38d4cff8f9d1bb8983f99ef` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:45.506428 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a037c0-5091-4ead-8cb4-442ab955ca60 admin admin] RBAC: Authorizing `identity:create_grant(project_id=983f56f36a63477690fe70282ae57787, user_id=e632e4d5279c4046b2c0976449152df4, role_id=580aa7dbb38d4cff8f9d1bb8983f99ef)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:16:45.509830 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a037c0-5091-4ead-8cb4-442ab955ca60 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:16:45.582687 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa35bda0-88b4-4d34-b812-abe53e2b92e9 admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:16:45.588618 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3e36ed-ebbb-40d3-b9d7-1761cbb03f92 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abe9d8c9291845dfa4ef55034e5c9e32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:16:45.589358 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3e36ed-ebbb-40d3-b9d7-1761cbb03f92 service placement] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:45.589794 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3e36ed-ebbb-40d3-b9d7-1761cbb03f92 service placement] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:45.590235 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3e36ed-ebbb-40d3-b9d7-1761cbb03f92 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:45.592174 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-fe3e36ed-ebbb-40d3-b9d7-1761cbb03f92 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:16:45.606628 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3e36ed-ebbb-40d3-b9d7-1761cbb03f92 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:16:45.608408 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3e36ed-ebbb-40d3-b9d7-1761cbb03f92 service placement] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:16:45.619819 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 873/1751] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:16:45 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:16:45.621255 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3e36ed-ebbb-40d3-b9d7-1761cbb03f92 service placement] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:16:45.633459 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3e36ed-ebbb-40d3-b9d7-1761cbb03f92 service placement] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:16:45.639010 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfebe9c-028e-4dc8-8ed9-fcb4751f236d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:16:45.639949 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfebe9c-028e-4dc8-8ed9-fcb4751f236d admin admin] REQUEST_METHOD: `PUT` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:45.640272 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfebe9c-028e-4dc8-8ed9-fcb4751f236d admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:45.640750 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfebe9c-028e-4dc8-8ed9-fcb4751f236d admin admin] PATH_INFO: `/v3/projects/7e2a54e00e374c1d9f11ce2269ee59fa/users/69d9357064b14d9d8242fc3051db7cd2/roles/49e47cd157d743f6905f1db07ec878a7` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:45.661023 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bfebe9c-028e-4dc8-8ed9-fcb4751f236d admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e2a54e00e374c1d9f11ce2269ee59fa, user_id=69d9357064b14d9d8242fc3051db7cd2, role_id=49e47cd157d743f6905f1db07ec878a7)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:16:45.663860 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bfebe9c-028e-4dc8-8ed9-fcb4751f236d admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:16:45.724634 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 879/1752] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Aug 24 06:16:45 2022] PUT /identity/v3/projects/983f56f36a63477690fe70282ae57787/users/e632e4d5279c4046b2c0976449152df4/roles/580aa7dbb38d4cff8f9d1bb8983f99ef => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:16:45.729086 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a037c0-5091-4ead-8cb4-442ab955ca60 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:16:45.739532 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-10a037c0-5091-4ead-8cb4-442ab955ca60 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:16:45.791595 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 874/1753] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Aug 24 06:16:45 2022] PUT /identity/v3/projects/7e2a54e00e374c1d9f11ce2269ee59fa/users/69d9357064b14d9d8242fc3051db7cd2/roles/49e47cd157d743f6905f1db07ec878a7 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:16:45.791595 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2377c6-6f86-455b-8430-4ad4c1bb85a0 None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:45.791595 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2377c6-6f86-455b-8430-4ad4c1bb85a0 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:45.791595 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2377c6-6f86-455b-8430-4ad4c1bb85a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:45.894294 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-7c2377c6-6f86-455b-8430-4ad4c1bb85a0 None None] MFA Rules not processed for user `e632e4d5279c4046b2c0976449152df4`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:16:45.899811 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-7c2377c6-6f86-455b-8430-4ad4c1bb85a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:16:45.942418 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a037c0-5091-4ead-8cb4-442ab955ca60 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:16:46.038385 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:16:46.039046 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 875/1754] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:16:45 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:16:46.039046 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2377c6-6f86-455b-8430-4ad4c1bb85a0 None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:16:46.073974 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e7c612-d6c9-42f5-8f9c-7c140139d9b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:16:46.075022 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e7c612-d6c9-42f5-8f9c-7c140139d9b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:46.075488 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e7c612-d6c9-42f5-8f9c-7c140139d9b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:46.078055 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e7c612-d6c9-42f5-8f9c-7c140139d9b3 admin admin] PATH_INFO: `/v3/projects/00d93c0b6f064311b732714aa91a96bb` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:46.080551 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e7c612-d6c9-42f5-8f9c-7c140139d9b3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=00d93c0b6f064311b732714aa91a96bb)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:16:46.087290 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e7c612-d6c9-42f5-8f9c-7c140139d9b3 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:16:46.163786 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2377c6-6f86-455b-8430-4ad4c1bb85a0 None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:16:46.166544 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef5941a-0a6f-403d-970f-9d67a81f6b84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:16:46.167354 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef5941a-0a6f-403d-970f-9d67a81f6b84 admin admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:46.167737 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef5941a-0a6f-403d-970f-9d67a81f6b84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:46.168108 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef5941a-0a6f-403d-970f-9d67a81f6b84 admin admin] PATH_INFO: `/v3/roles` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:46.170106 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef5941a-0a6f-403d-970f-9d67a81f6b84 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:16:46.171857 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef5941a-0a6f-403d-970f-9d67a81f6b84 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:16:46.192000 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 876/1755] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Aug 24 06:16:46 2022] GET /identity/v3/roles => generated 1420 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:16:46.198547 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef5941a-0a6f-403d-970f-9d67a81f6b84 admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:16:46.204515 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef5941a-0a6f-403d-970f-9d67a81f6b84 admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:16:46.210568 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950572b1-b926-4fe4-ad42-8941e1f62677 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:16:46.211250 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950572b1-b926-4fe4-ad42-8941e1f62677 admin admin] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:46.211495 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950572b1-b926-4fe4-ad42-8941e1f62677 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:46.211922 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950572b1-b926-4fe4-ad42-8941e1f62677 admin admin] PATH_INFO: `/v3/projects` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:46.212728 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950572b1-b926-4fe4-ad42-8941e1f62677 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:16:46.215456 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950572b1-b926-4fe4-ad42-8941e1f62677 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:16:46.281289 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-e3e7c612-d6c9-42f5-8f9c-7c140139d9b3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '00d93c0b6f064311b732714aa91a96bb', 'request_id': 'req-e3e7c612-d6c9-42f5-8f9c-7c140139d9b3'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:16:46.289368 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-e3e7c612-d6c9-42f5-8f9c-7c140139d9b3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '00d93c0b6f064311b732714aa91a96bb', 'request_id': 'req-e3e7c612-d6c9-42f5-8f9c-7c140139d9b3'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:16:46.296471 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-e3e7c612-d6c9-42f5-8f9c-7c140139d9b3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '00d93c0b6f064311b732714aa91a96bb', 'request_id': 'req-e3e7c612-d6c9-42f5-8f9c-7c140139d9b3'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:16:46.347713 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 877/1756] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Aug 24 06:16:46 2022] POST /identity/v3/projects => generated 367 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:16:46.350425 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a6c2eb-45b2-4766-ab43-5add9153bb91 None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:46.353083 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a6c2eb-45b2-4766-ab43-5add9153bb91 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:46.353386 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a6c2eb-45b2-4766-ab43-5add9153bb91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:46.404337 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-e6a6c2eb-45b2-4766-ab43-5add9153bb91 None None] MFA Rules not processed for user `5059723f417445fa934ecfe3e50d0197`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:16:46.405659 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-e6a6c2eb-45b2-4766-ab43-5add9153bb91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:16:46.437387 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-e3e7c612-d6c9-42f5-8f9c-7c140139d9b3 admin admin] The token cache is being invalidate because project 00d93c0b6f064311b732714aa91a96bb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81852) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:16:46.439542 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-e3e7c612-d6c9-42f5-8f9c-7c140139d9b3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:16:46.440628 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-e3e7c612-d6c9-42f5-8f9c-7c140139d9b3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:16:46.453861 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 880/1757] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Aug 24 06:16:45 2022] DELETE /identity/v3/projects/00d93c0b6f064311b732714aa91a96bb => generated 0 bytes in 726 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:16:46.457110 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e7c612-d6c9-42f5-8f9c-7c140139d9b3 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:16:46.458696 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-e3e7c612-d6c9-42f5-8f9c-7c140139d9b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:16:46.541460 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 878/1758] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:16:46 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:16:46.542468 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a6c2eb-45b2-4766-ab43-5add9153bb91 None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:16:46.545265 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-e6a6c2eb-45b2-4766-ab43-5add9153bb91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:16:46.627911 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e7c612-d6c9-42f5-8f9c-7c140139d9b3 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:16:46.712004 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a6c2eb-45b2-4766-ab43-5add9153bb91 None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:16:46.734527 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1121054b-168e-4645-82f8-c02c23094749 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:16:46.738294 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1121054b-168e-4645-82f8-c02c23094749 admin admin] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:46.738863 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1121054b-168e-4645-82f8-c02c23094749 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:46.740773 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1121054b-168e-4645-82f8-c02c23094749 admin admin] PATH_INFO: `/v3/users` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:46.741410 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1121054b-168e-4645-82f8-c02c23094749 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:16:46.746577 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1121054b-168e-4645-82f8-c02c23094749 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:16:46.772165 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b77ccd-903b-4bcf-b8bc-68df585f945d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:16:46.773201 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b77ccd-903b-4bcf-b8bc-68df585f945d admin admin] REQUEST_METHOD: `PUT` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:46.773599 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b77ccd-903b-4bcf-b8bc-68df585f945d admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:46.773995 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b77ccd-903b-4bcf-b8bc-68df585f945d admin admin] PATH_INFO: `/v3/projects/7e2a54e00e374c1d9f11ce2269ee59fa/users/69d9357064b14d9d8242fc3051db7cd2/roles/49e47cd157d743f6905f1db07ec878a7` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:46.776093 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b77ccd-903b-4bcf-b8bc-68df585f945d admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e2a54e00e374c1d9f11ce2269ee59fa, user_id=69d9357064b14d9d8242fc3051db7cd2, role_id=49e47cd157d743f6905f1db07ec878a7)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:16:46.779144 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b77ccd-903b-4bcf-b8bc-68df585f945d admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:16:46.796629 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 881/1759] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Aug 24 06:16:46 2022] PUT /identity/v3/projects/7e2a54e00e374c1d9f11ce2269ee59fa/users/69d9357064b14d9d8242fc3051db7cd2/roles/49e47cd157d743f6905f1db07ec878a7 => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:16:46.796629 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b77ccd-903b-4bcf-b8bc-68df585f945d admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:16:46.796629 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-94b77ccd-903b-4bcf-b8bc-68df585f945d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:16:46.862484 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 879/1760] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Aug 24 06:16:46 2022] POST /identity/v3/users => generated 290 bytes in 320 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:16:46.863838 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6819e34-170d-432c-b471-45c6ea07e72a None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:46.864247 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6819e34-170d-432c-b471-45c6ea07e72a None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:46.880620 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6819e34-170d-432c-b471-45c6ea07e72a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:46.971735 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b77ccd-903b-4bcf-b8bc-68df585f945d admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:16:46.977565 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb8eadd-52c9-448a-9386-0d0eafd3c916 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:16:46.978587 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb8eadd-52c9-448a-9386-0d0eafd3c916 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:46.979077 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb8eadd-52c9-448a-9386-0d0eafd3c916 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:46.979515 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb8eadd-52c9-448a-9386-0d0eafd3c916 admin admin] PATH_INFO: `/v3/users/16d019e12c164a208003ba5710c7ea15` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:46.982038 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb8eadd-52c9-448a-9386-0d0eafd3c916 admin admin] RBAC: Authorizing `identity:delete_user(user_id=16d019e12c164a208003ba5710c7ea15)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:16:46.984324 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-f6819e34-170d-432c-b471-45c6ea07e72a None None] MFA Rules not processed for user `69d9357064b14d9d8242fc3051db7cd2`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:16:46.984570 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb8eadd-52c9-448a-9386-0d0eafd3c916 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:16:46.994038 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-f6819e34-170d-432c-b471-45c6ea07e72a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:16:47.093028 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:16:47.202439 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 880/1761] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:16:46 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 335 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:16:47.202439 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6819e34-170d-432c-b471-45c6ea07e72a None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:16:47.215148 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6819e34-170d-432c-b471-45c6ea07e72a None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:16:47.220195 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fe60d2-5429-4004-90c1-24db8344228a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:16:47.220878 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fe60d2-5429-4004-90c1-24db8344228a admin admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:47.221157 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fe60d2-5429-4004-90c1-24db8344228a admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:47.221425 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fe60d2-5429-4004-90c1-24db8344228a admin admin] PATH_INFO: `/v3/roles` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:47.222106 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52fe60d2-5429-4004-90c1-24db8344228a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:16:47.223914 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52fe60d2-5429-4004-90c1-24db8344228a admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:16:47.240249 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 881/1762] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Aug 24 06:16:47 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:16:47.240249 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fe60d2-5429-4004-90c1-24db8344228a admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:16:47.241476 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-52fe60d2-5429-4004-90c1-24db8344228a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:16:47.263061 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-1cb8eadd-52c9-448a-9386-0d0eafd3c916 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '16d019e12c164a208003ba5710c7ea15'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:16:47.280661 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-1cb8eadd-52c9-448a-9386-0d0eafd3c916 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '16d019e12c164a208003ba5710c7ea15'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:16:47.283736 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-1cb8eadd-52c9-448a-9386-0d0eafd3c916 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '16d019e12c164a208003ba5710c7ea15'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:16:47.358824 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fe60d2-5429-4004-90c1-24db8344228a admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:16:47.365037 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2961988-7c4f-46d2-8485-3bb64553c451 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c4def55ac0e48b29836bc5c64bc7402', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:16:47.365962 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2961988-7c4f-46d2-8485-3bb64553c451 service cinder] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:47.366332 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2961988-7c4f-46d2-8485-3bb64553c451 service cinder] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:47.366713 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2961988-7c4f-46d2-8485-3bb64553c451 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:47.377051 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2961988-7c4f-46d2-8485-3bb64553c451 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:16:47.378711 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2961988-7c4f-46d2-8485-3bb64553c451 service cinder] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:16:47.388963 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 882/1763] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:16:47 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:16:47.389913 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2961988-7c4f-46d2-8485-3bb64553c451 service cinder] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:16:47.407612 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-1cb8eadd-52c9-448a-9386-0d0eafd3c916 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '16d019e12c164a208003ba5710c7ea15'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:16:47.419781 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2961988-7c4f-46d2-8485-3bb64553c451 service cinder] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:16:47.433502 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-1cb8eadd-52c9-448a-9386-0d0eafd3c916 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '16d019e12c164a208003ba5710c7ea15'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:16:47.437484 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0336b778-6ffb-484c-956b-2d83cd73fc26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:16:47.438985 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0336b778-6ffb-484c-956b-2d83cd73fc26 admin admin] REQUEST_METHOD: `PUT` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:47.440767 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0336b778-6ffb-484c-956b-2d83cd73fc26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:47.441160 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0336b778-6ffb-484c-956b-2d83cd73fc26 admin admin] PATH_INFO: `/v3/projects/89ed44a904d84eceae80b44443ae6413/users/aa4f92933e8541ab91d6f074572fa6ee/roles/49e47cd157d743f6905f1db07ec878a7` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:47.469762 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 882/1764] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Aug 24 06:16:46 2022] DELETE /identity/v3/users/16d019e12c164a208003ba5710c7ea15 => generated 0 bytes in 671 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:16:47.485841 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0336b778-6ffb-484c-956b-2d83cd73fc26 admin admin] RBAC: Authorizing `identity:create_grant(project_id=89ed44a904d84eceae80b44443ae6413, user_id=aa4f92933e8541ab91d6f074572fa6ee, role_id=49e47cd157d743f6905f1db07ec878a7)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:16:47.488944 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0336b778-6ffb-484c-956b-2d83cd73fc26 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:16:47.513689 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb8eadd-52c9-448a-9386-0d0eafd3c916 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:16:47.517959 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-1cb8eadd-52c9-448a-9386-0d0eafd3c916 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:16:47.574660 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 883/1765] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Aug 24 06:16:47 2022] PUT /identity/v3/projects/89ed44a904d84eceae80b44443ae6413/users/aa4f92933e8541ab91d6f074572fa6ee/roles/49e47cd157d743f6905f1db07ec878a7 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:16:47.582731 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0336b778-6ffb-484c-956b-2d83cd73fc26 admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:16:47.585895 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-0336b778-6ffb-484c-956b-2d83cd73fc26 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:16:47.715488 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb8eadd-52c9-448a-9386-0d0eafd3c916 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:16:47.776201 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0336b778-6ffb-484c-956b-2d83cd73fc26 admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:16:47.788761 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd67ce11-a337-4d18-b79c-ef5760f62d55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:16:47.789575 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd67ce11-a337-4d18-b79c-ef5760f62d55 admin admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:47.791120 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd67ce11-a337-4d18-b79c-ef5760f62d55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:47.791579 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd67ce11-a337-4d18-b79c-ef5760f62d55 admin admin] PATH_INFO: `/v3/roles` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:47.794071 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd67ce11-a337-4d18-b79c-ef5760f62d55 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:16:47.797608 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd67ce11-a337-4d18-b79c-ef5760f62d55 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:16:47.824551 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 884/1766] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Aug 24 06:16:47 2022] GET /identity/v3/roles => generated 1420 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:16:47.828701 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd67ce11-a337-4d18-b79c-ef5760f62d55 admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:16:47.850463 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd67ce11-a337-4d18-b79c-ef5760f62d55 admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:16:47.865068 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3389631a-59c8-4a45-85c8-4c2c000cc540 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:16:47.865878 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3389631a-59c8-4a45-85c8-4c2c000cc540 admin admin] REQUEST_METHOD: `PUT` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:47.866424 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3389631a-59c8-4a45-85c8-4c2c000cc540 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:47.866808 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3389631a-59c8-4a45-85c8-4c2c000cc540 admin admin] PATH_INFO: `/v3/projects/89ed44a904d84eceae80b44443ae6413/users/aa4f92933e8541ab91d6f074572fa6ee/roles/49e47cd157d743f6905f1db07ec878a7` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:47.870733 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cfb817-9b62-4558-b8e3-da4d37c1f8eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:16:47.871846 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3389631a-59c8-4a45-85c8-4c2c000cc540 admin admin] RBAC: Authorizing `identity:create_grant(project_id=89ed44a904d84eceae80b44443ae6413, user_id=aa4f92933e8541ab91d6f074572fa6ee, role_id=49e47cd157d743f6905f1db07ec878a7)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:16:47.871946 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cfb817-9b62-4558-b8e3-da4d37c1f8eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:47.872818 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cfb817-9b62-4558-b8e3-da4d37c1f8eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:47.873164 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cfb817-9b62-4558-b8e3-da4d37c1f8eb admin admin] PATH_INFO: `/v3/users/1efc05afaa684f79a5fd8ed9f4d53ff8` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:47.874409 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07cfb817-9b62-4558-b8e3-da4d37c1f8eb admin admin] RBAC: Authorizing `identity:delete_user(user_id=1efc05afaa684f79a5fd8ed9f4d53ff8)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:16:47.876842 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07cfb817-9b62-4558-b8e3-da4d37c1f8eb admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:16:47.882446 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3389631a-59c8-4a45-85c8-4c2c000cc540 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:16:47.897683 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 885/1767] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Aug 24 06:16:47 2022] PUT /identity/v3/projects/89ed44a904d84eceae80b44443ae6413/users/aa4f92933e8541ab91d6f074572fa6ee/roles/49e47cd157d743f6905f1db07ec878a7 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:16:47.908572 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199a6dd0-9d1a-4f05-b857-3a50dde20791 None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:47.911227 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199a6dd0-9d1a-4f05-b857-3a50dde20791 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:47.911503 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199a6dd0-9d1a-4f05-b857-3a50dde20791 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:47.976583 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-199a6dd0-9d1a-4f05-b857-3a50dde20791 None None] MFA Rules not processed for user `aa4f92933e8541ab91d6f074572fa6ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:16:47.986349 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-199a6dd0-9d1a-4f05-b857-3a50dde20791 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:16:48.025453 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:16:48.164200 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-07cfb817-9b62-4558-b8e3-da4d37c1f8eb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1efc05afaa684f79a5fd8ed9f4d53ff8'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:16:48.173238 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-07cfb817-9b62-4558-b8e3-da4d37c1f8eb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1efc05afaa684f79a5fd8ed9f4d53ff8'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:16:48.174131 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-07cfb817-9b62-4558-b8e3-da4d37c1f8eb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1efc05afaa684f79a5fd8ed9f4d53ff8'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:16:48.217708 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 886/1768] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:16:47 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 301 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 06:16:48.229997 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199a6dd0-9d1a-4f05-b857-3a50dde20791 None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:16:48.232151 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-199a6dd0-9d1a-4f05-b857-3a50dde20791 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:16:48.306281 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-07cfb817-9b62-4558-b8e3-da4d37c1f8eb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1efc05afaa684f79a5fd8ed9f4d53ff8'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:16:48.328583 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199a6dd0-9d1a-4f05-b857-3a50dde20791 None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:16:48.338464 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df4cf38-2279-4658-bb88-a7f541d6ef5a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c4def55ac0e48b29836bc5c64bc7402', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:16:48.339752 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df4cf38-2279-4658-bb88-a7f541d6ef5a service cinder] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:48.340042 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df4cf38-2279-4658-bb88-a7f541d6ef5a service cinder] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:48.340318 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df4cf38-2279-4658-bb88-a7f541d6ef5a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:48.350413 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-07cfb817-9b62-4558-b8e3-da4d37c1f8eb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1efc05afaa684f79a5fd8ed9f4d53ff8'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:16:48.353353 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df4cf38-2279-4658-bb88-a7f541d6ef5a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:16:48.354898 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df4cf38-2279-4658-bb88-a7f541d6ef5a service cinder] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:16:48.380108 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 887/1769] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:16:48 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:16:48.381521 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df4cf38-2279-4658-bb88-a7f541d6ef5a service cinder] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:16:48.383046 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-8df4cf38-2279-4658-bb88-a7f541d6ef5a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:16:48.392738 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 883/1770] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Aug 24 06:16:47 2022] DELETE /identity/v3/users/1efc05afaa684f79a5fd8ed9f4d53ff8 => generated 0 bytes in 877 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:16:48.397694 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980a4821-65a1-49ee-9a0c-0bf13666bb88 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:48.397987 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980a4821-65a1-49ee-9a0c-0bf13666bb88 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:48.398265 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980a4821-65a1-49ee-9a0c-0bf13666bb88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:48.459154 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-980a4821-65a1-49ee-9a0c-0bf13666bb88 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:16:48.462401 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-980a4821-65a1-49ee-9a0c-0bf13666bb88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:16:48.465415 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df4cf38-2279-4658-bb88-a7f541d6ef5a service cinder] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:16:48.544566 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8f9a82-d3f0-40f6-8377-e9761a992111 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:16:48.546360 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8f9a82-d3f0-40f6-8377-e9761a992111 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:48.546652 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8f9a82-d3f0-40f6-8377-e9761a992111 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:48.548816 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8f9a82-d3f0-40f6-8377-e9761a992111 admin admin] PATH_INFO: `/v3/projects/b48751354827457e8ba7f0e71362f0de` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:48.550945 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8f9a82-d3f0-40f6-8377-e9761a992111 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b48751354827457e8ba7f0e71362f0de)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:16:48.558892 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8f9a82-d3f0-40f6-8377-e9761a992111 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:16:48.592118 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 884/1771] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:16:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 06:16:48.596045 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:16:48.607353 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980a4821-65a1-49ee-9a0c-0bf13666bb88 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:16:48.619310 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-980a4821-65a1-49ee-9a0c-0bf13666bb88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:16:48.699754 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980a4821-65a1-49ee-9a0c-0bf13666bb88 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:16:48.705024 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4668214f-3adc-49fa-ad31-4622c89a401d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bb6c79db75e4d169331c02133d1ec32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:16:48.706880 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4668214f-3adc-49fa-ad31-4622c89a401d service neutron] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:48.707384 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4668214f-3adc-49fa-ad31-4622c89a401d service neutron] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:48.708085 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4668214f-3adc-49fa-ad31-4622c89a401d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:48.736727 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4668214f-3adc-49fa-ad31-4622c89a401d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:16:48.745651 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4668214f-3adc-49fa-ad31-4622c89a401d service neutron] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:16:48.770924 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 885/1772] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:16:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 06:16:48.783868 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-4d8f9a82-d3f0-40f6-8377-e9761a992111 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b48751354827457e8ba7f0e71362f0de', 'request_id': 'req-4d8f9a82-d3f0-40f6-8377-e9761a992111'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:16:48.824361 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-4d8f9a82-d3f0-40f6-8377-e9761a992111 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b48751354827457e8ba7f0e71362f0de', 'request_id': 'req-4d8f9a82-d3f0-40f6-8377-e9761a992111'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:16:48.838901 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455d815c-ddd7-4077-a818-36f89642c38c None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:48.839199 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455d815c-ddd7-4077-a818-36f89642c38c None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:48.839467 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455d815c-ddd7-4077-a818-36f89642c38c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:48.864805 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-455d815c-ddd7-4077-a818-36f89642c38c None None] MFA Rules not processed for user `6abfd0a28f9e482badae7018f1b7f9a1`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:16:48.866874 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-455d815c-ddd7-4077-a818-36f89642c38c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:16:48.868854 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-4d8f9a82-d3f0-40f6-8377-e9761a992111 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b48751354827457e8ba7f0e71362f0de', 'request_id': 'req-4d8f9a82-d3f0-40f6-8377-e9761a992111'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:16:48.983538 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-4d8f9a82-d3f0-40f6-8377-e9761a992111 admin admin] The token cache is being invalidate because project b48751354827457e8ba7f0e71362f0de was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81851) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:16:48.983847 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-4d8f9a82-d3f0-40f6-8377-e9761a992111 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:16:49.000151 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-4d8f9a82-d3f0-40f6-8377-e9761a992111 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:16:49.006161 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 888/1773] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Aug 24 06:16:48 2022] DELETE /identity/v3/projects/b48751354827457e8ba7f0e71362f0de => generated 0 bytes in 621 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:16:49.007850 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf41e2a-adb5-4474-976f-dc00b6adda4b None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:49.008142 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf41e2a-adb5-4474-976f-dc00b6adda4b None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:49.008421 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf41e2a-adb5-4474-976f-dc00b6adda4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:49.017344 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 886/1774] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:16:48 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:16:49.038837 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-5cf41e2a-adb5-4474-976f-dc00b6adda4b None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:16:49.040228 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-5cf41e2a-adb5-4474-976f-dc00b6adda4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:16:49.298645 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 889/1775] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:16:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 292 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:16:49.309913 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3233a4be-2d3f-405f-aaf7-0174052e3f0f None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:49.310470 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3233a4be-2d3f-405f-aaf7-0174052e3f0f None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:49.310470 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3233a4be-2d3f-405f-aaf7-0174052e3f0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:49.376385 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-3233a4be-2d3f-405f-aaf7-0174052e3f0f None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:16:49.379105 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-3233a4be-2d3f-405f-aaf7-0174052e3f0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:16:49.404523 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 887/1776] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:16:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:16:49.416129 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf41e2a-adb5-4474-976f-dc00b6adda4b None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:16:49.470322 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf41e2a-adb5-4474-976f-dc00b6adda4b None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:16:49.481349 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b67afdc-61fd-4c88-afdd-6b8e5fdd9898 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:16:49.483283 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b67afdc-61fd-4c88-afdd-6b8e5fdd9898 admin admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:49.484618 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b67afdc-61fd-4c88-afdd-6b8e5fdd9898 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:49.485046 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b67afdc-61fd-4c88-afdd-6b8e5fdd9898 admin admin] PATH_INFO: `/v3/domains` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:49.487534 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b67afdc-61fd-4c88-afdd-6b8e5fdd9898 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:16:49.488704 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b67afdc-61fd-4c88-afdd-6b8e5fdd9898 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:16:49.493065 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b67afdc-61fd-4c88-afdd-6b8e5fdd9898 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:16:49.523906 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 890/1777] 10.222.0.62 () {58 vars in 1153 bytes} [Wed Aug 24 06:16:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:16:49.528712 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3233a4be-2d3f-405f-aaf7-0174052e3f0f None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:16:49.543631 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3233a4be-2d3f-405f-aaf7-0174052e3f0f None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:16:49.555684 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6a138c-1347-4408-abe0-1b928a2f5e5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:16:49.559215 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6a138c-1347-4408-abe0-1b928a2f5e5e admin admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:49.561621 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6a138c-1347-4408-abe0-1b928a2f5e5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:49.561937 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6a138c-1347-4408-abe0-1b928a2f5e5e admin admin] PATH_INFO: `/v3/projects` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:49.563906 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6a138c-1347-4408-abe0-1b928a2f5e5e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:16:49.569572 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6a138c-1347-4408-abe0-1b928a2f5e5e admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:16:49.739048 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 888/1778] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Aug 24 06:16:49 2022] POST /identity/v3/projects => generated 363 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:16:49.744123 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b67afdc-61fd-4c88-afdd-6b8e5fdd9898 admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:16:49.763449 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b67afdc-61fd-4c88-afdd-6b8e5fdd9898 admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:16:49.907568 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1adb7a-a7d3-4a7a-9cee-9548b6aa8c9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:16:49.908258 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1adb7a-a7d3-4a7a-9cee-9548b6aa8c9e admin admin] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:49.908557 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1adb7a-a7d3-4a7a-9cee-9548b6aa8c9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:49.909346 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1adb7a-a7d3-4a7a-9cee-9548b6aa8c9e admin admin] PATH_INFO: `/v3/users` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:49.910152 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1adb7a-a7d3-4a7a-9cee-9548b6aa8c9e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:16:49.913350 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1adb7a-a7d3-4a7a-9cee-9548b6aa8c9e admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:16:50.016284 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:16:50.025600 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6a138c-1347-4408-abe0-1b928a2f5e5e admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:16:50.027686 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-9b6a138c-1347-4408-abe0-1b928a2f5e5e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:16:50.068485 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6a138c-1347-4408-abe0-1b928a2f5e5e admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:16:50.084400 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8de46e0-6b2c-4de1-98ea-d79929be308b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:16:50.086366 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8de46e0-6b2c-4de1-98ea-d79929be308b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:50.086850 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8de46e0-6b2c-4de1-98ea-d79929be308b admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:50.087137 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8de46e0-6b2c-4de1-98ea-d79929be308b admin admin] PATH_INFO: `/v3/projects/3b7c95ff81674cbebc48223c0fc09516` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:50.090934 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8de46e0-6b2c-4de1-98ea-d79929be308b admin admin] RBAC: Authorizing `identity:delete_project(project_id=3b7c95ff81674cbebc48223c0fc09516)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:16:50.095156 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8de46e0-6b2c-4de1-98ea-d79929be308b admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:16:50.142080 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 891/1779] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Aug 24 06:16:49 2022] POST /identity/v3/users => generated 282 bytes in 399 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:16:50.152627 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1adb7a-a7d3-4a7a-9cee-9548b6aa8c9e admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:16:50.169976 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1adb7a-a7d3-4a7a-9cee-9548b6aa8c9e admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:16:50.179803 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa7bd21-2118-4e5d-91f2-346f7e76e336 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:16:50.181135 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa7bd21-2118-4e5d-91f2-346f7e76e336 admin admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:50.181413 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa7bd21-2118-4e5d-91f2-346f7e76e336 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:50.181686 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa7bd21-2118-4e5d-91f2-346f7e76e336 admin admin] PATH_INFO: `/v3/roles` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:50.183073 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa7bd21-2118-4e5d-91f2-346f7e76e336 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:16:50.186876 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa7bd21-2118-4e5d-91f2-346f7e76e336 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:16:50.209179 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 892/1780] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Aug 24 06:16:50 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 06:16:50.212871 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa7bd21-2118-4e5d-91f2-346f7e76e336 admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:16:50.231774 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa7bd21-2118-4e5d-91f2-346f7e76e336 admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:16:50.247550 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949232f8-044d-4402-b322-9a9d166d6285 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:16:50.247550 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949232f8-044d-4402-b322-9a9d166d6285 admin admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:50.247550 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949232f8-044d-4402-b322-9a9d166d6285 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:50.247550 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949232f8-044d-4402-b322-9a9d166d6285 admin admin] PATH_INFO: `/v3/roles` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:50.248524 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949232f8-044d-4402-b322-9a9d166d6285 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:16:50.248524 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949232f8-044d-4402-b322-9a9d166d6285 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:16:50.274721 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 893/1781] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Aug 24 06:16:50 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 06:16:50.277895 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949232f8-044d-4402-b322-9a9d166d6285 admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:16:50.297265 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949232f8-044d-4402-b322-9a9d166d6285 admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:16:50.311888 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d839dd34-6472-40bb-9969-99acc333880d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:16:50.312596 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d839dd34-6472-40bb-9969-99acc333880d admin admin] REQUEST_METHOD: `PUT` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:50.312876 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d839dd34-6472-40bb-9969-99acc333880d admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:50.313152 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d839dd34-6472-40bb-9969-99acc333880d admin admin] PATH_INFO: `/v3/projects/c779507bb66d449e8550c9eb45500474/users/15bda6a0e79c4a7e89f9b9527b9cd60a/roles/580aa7dbb38d4cff8f9d1bb8983f99ef` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:50.349154 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-c8de46e0-6b2c-4de1-98ea-d79929be308b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3b7c95ff81674cbebc48223c0fc09516', 'request_id': 'req-c8de46e0-6b2c-4de1-98ea-d79929be308b'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:16:50.355725 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d839dd34-6472-40bb-9969-99acc333880d admin admin] RBAC: Authorizing `identity:create_grant(project_id=c779507bb66d449e8550c9eb45500474, user_id=15bda6a0e79c4a7e89f9b9527b9cd60a, role_id=580aa7dbb38d4cff8f9d1bb8983f99ef)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:16:50.359158 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d839dd34-6472-40bb-9969-99acc333880d admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:16:50.367210 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-c8de46e0-6b2c-4de1-98ea-d79929be308b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3b7c95ff81674cbebc48223c0fc09516', 'request_id': 'req-c8de46e0-6b2c-4de1-98ea-d79929be308b'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:16:50.378336 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-c8de46e0-6b2c-4de1-98ea-d79929be308b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3b7c95ff81674cbebc48223c0fc09516', 'request_id': 'req-c8de46e0-6b2c-4de1-98ea-d79929be308b'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:16:50.431746 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:16:50.452078 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 894/1782] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Aug 24 06:16:50 2022] PUT /identity/v3/projects/c779507bb66d449e8550c9eb45500474/users/15bda6a0e79c4a7e89f9b9527b9cd60a/roles/580aa7dbb38d4cff8f9d1bb8983f99ef => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:16:50.462506 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1af2d90-84ac-40e4-84af-c900ad8bc3de None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:50.462836 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1af2d90-84ac-40e4-84af-c900ad8bc3de None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:50.463183 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1af2d90-84ac-40e4-84af-c900ad8bc3de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:50.494191 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-c8de46e0-6b2c-4de1-98ea-d79929be308b admin admin] The token cache is being invalidate because project 3b7c95ff81674cbebc48223c0fc09516 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81852) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:16:50.494705 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-c8de46e0-6b2c-4de1-98ea-d79929be308b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:16:50.496771 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-c8de46e0-6b2c-4de1-98ea-d79929be308b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:16:50.500318 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 889/1783] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Aug 24 06:16:50 2022] DELETE /identity/v3/projects/3b7c95ff81674cbebc48223c0fc09516 => generated 0 bytes in 475 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 06:16:50.502024 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-d1af2d90-84ac-40e4-84af-c900ad8bc3de None None] MFA Rules not processed for user `15bda6a0e79c4a7e89f9b9527b9cd60a`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:16:50.503397 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-d1af2d90-84ac-40e4-84af-c900ad8bc3de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:16:50.505252 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8de46e0-6b2c-4de1-98ea-d79929be308b admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:16:50.506984 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-c8de46e0-6b2c-4de1-98ea-d79929be308b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:16:50.680899 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 895/1784] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:16:50 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:16:50.682028 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1af2d90-84ac-40e4-84af-c900ad8bc3de None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:16:50.687246 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-d1af2d90-84ac-40e4-84af-c900ad8bc3de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:16:50.748214 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8de46e0-6b2c-4de1-98ea-d79929be308b admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:16:50.915066 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1af2d90-84ac-40e4-84af-c900ad8bc3de None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:16:50.936295 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b065fc4-7c67-4602-9b25-06da819984dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:16:50.939491 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b065fc4-7c67-4602-9b25-06da819984dd admin admin] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:50.941803 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b065fc4-7c67-4602-9b25-06da819984dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:50.944235 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b065fc4-7c67-4602-9b25-06da819984dd admin admin] PATH_INFO: `/v3/projects` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:50.945118 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b065fc4-7c67-4602-9b25-06da819984dd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:16:50.982014 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b065fc4-7c67-4602-9b25-06da819984dd admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:16:50.994889 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d749c97-3ce1-4bb2-b720-8418ce66b278 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:16:50.996736 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d749c97-3ce1-4bb2-b720-8418ce66b278 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:50.999305 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d749c97-3ce1-4bb2-b720-8418ce66b278 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:51.001227 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d749c97-3ce1-4bb2-b720-8418ce66b278 admin admin] PATH_INFO: `/v3/users/69d9357064b14d9d8242fc3051db7cd2` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:51.008638 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d749c97-3ce1-4bb2-b720-8418ce66b278 admin admin] RBAC: Authorizing `identity:delete_user(user_id=69d9357064b14d9d8242fc3051db7cd2)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:16:51.016229 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d749c97-3ce1-4bb2-b720-8418ce66b278 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:16:51.145810 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 896/1785] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Aug 24 06:16:50 2022] POST /identity/v3/projects => generated 365 bytes in 464 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:16:51.156530 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b065fc4-7c67-4602-9b25-06da819984dd admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:16:51.316920 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b065fc4-7c67-4602-9b25-06da819984dd admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:16:51.329152 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdfba1a-0da7-41dc-912d-25f768c0a64b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:16:51.329785 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdfba1a-0da7-41dc-912d-25f768c0a64b admin admin] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:51.330085 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdfba1a-0da7-41dc-912d-25f768c0a64b admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:51.330331 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdfba1a-0da7-41dc-912d-25f768c0a64b admin admin] PATH_INFO: `/v3/users` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:51.331183 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bdfba1a-0da7-41dc-912d-25f768c0a64b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:16:51.333365 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bdfba1a-0da7-41dc-912d-25f768c0a64b admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:16:51.358508 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-6d749c97-3ce1-4bb2-b720-8418ce66b278 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '69d9357064b14d9d8242fc3051db7cd2'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:16:51.393930 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-6d749c97-3ce1-4bb2-b720-8418ce66b278 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '69d9357064b14d9d8242fc3051db7cd2'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:16:51.395249 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-6d749c97-3ce1-4bb2-b720-8418ce66b278 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '69d9357064b14d9d8242fc3051db7cd2'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:16:51.476623 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 897/1786] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Aug 24 06:16:51 2022] POST /identity/v3/users => generated 289 bytes in 328 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:16:51.476623 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258237dd-c8c5-4e2f-81fb-f7ee434e4541 None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:51.477313 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258237dd-c8c5-4e2f-81fb-f7ee434e4541 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:51.478825 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258237dd-c8c5-4e2f-81fb-f7ee434e4541 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:51.550639 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-6d749c97-3ce1-4bb2-b720-8418ce66b278 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '69d9357064b14d9d8242fc3051db7cd2'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:16:51.578304 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-6d749c97-3ce1-4bb2-b720-8418ce66b278 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '69d9357064b14d9d8242fc3051db7cd2'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:16:51.591208 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 890/1787] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Aug 24 06:16:50 2022] DELETE /identity/v3/users/69d9357064b14d9d8242fc3051db7cd2 => generated 0 bytes in 1086 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:16:51.592590 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d749c97-3ce1-4bb2-b720-8418ce66b278 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:16:51.594562 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-6d749c97-3ce1-4bb2-b720-8418ce66b278 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:16:51.615348 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-258237dd-c8c5-4e2f-81fb-f7ee434e4541 None None] MFA Rules not processed for user `b82726a12436435cb931b986ef884c2e`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:16:51.616760 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-258237dd-c8c5-4e2f-81fb-f7ee434e4541 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:16:51.708634 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d749c97-3ce1-4bb2-b720-8418ce66b278 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:16:51.722780 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f20cc7-e536-4400-934a-ddf63a655453 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:16:51.723671 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f20cc7-e536-4400-934a-ddf63a655453 admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:51.724002 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f20cc7-e536-4400-934a-ddf63a655453 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:51.726460 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f20cc7-e536-4400-934a-ddf63a655453 admin admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:51.727989 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f20cc7-e536-4400-934a-ddf63a655453 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:16:51.731311 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f20cc7-e536-4400-934a-ddf63a655453 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:16:51.757255 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 891/1788] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Aug 24 06:16:51 2022] GET /identity/v3/roles => generated 1420 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:16:51.758770 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f20cc7-e536-4400-934a-ddf63a655453 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:16:51.792215 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f20cc7-e536-4400-934a-ddf63a655453 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:16:51.796128 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 898/1789] 10.222.0.62 () {58 vars in 1009 bytes} [Wed Aug 24 06:16:51 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 321 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:16:51.806980 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680405dc-76b5-4cb1-931a-8e36fe8b076d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:16:51.807722 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680405dc-76b5-4cb1-931a-8e36fe8b076d admin admin] REQUEST_METHOD: `PUT` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:51.808002 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680405dc-76b5-4cb1-931a-8e36fe8b076d admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:51.808289 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680405dc-76b5-4cb1-931a-8e36fe8b076d admin admin] PATH_INFO: `/v3/projects/5053dda49bb14b28b9357947315170b6/users/54a17a48f603457995acbec718b6a3b7/roles/49e47cd157d743f6905f1db07ec878a7` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:51.839642 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680405dc-76b5-4cb1-931a-8e36fe8b076d admin admin] RBAC: Authorizing `identity:create_grant(project_id=5053dda49bb14b28b9357947315170b6, user_id=54a17a48f603457995acbec718b6a3b7, role_id=49e47cd157d743f6905f1db07ec878a7)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:16:51.844272 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680405dc-76b5-4cb1-931a-8e36fe8b076d admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:16:51.940639 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258237dd-c8c5-4e2f-81fb-f7ee434e4541 None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:16:51.940841 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 892/1790] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Aug 24 06:16:51 2022] PUT /identity/v3/projects/5053dda49bb14b28b9357947315170b6/users/54a17a48f603457995acbec718b6a3b7/roles/49e47cd157d743f6905f1db07ec878a7 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:16:51.980766 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258237dd-c8c5-4e2f-81fb-f7ee434e4541 None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:16:52.101751 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:16:52.164537 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a618ae-842d-45c7-b527-0ca74b76124c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:16:52.166681 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a618ae-842d-45c7-b527-0ca74b76124c admin admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:52.167180 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a618ae-842d-45c7-b527-0ca74b76124c admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:52.167609 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a618ae-842d-45c7-b527-0ca74b76124c admin admin] PATH_INFO: `/v3/roles` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:52.173705 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a618ae-842d-45c7-b527-0ca74b76124c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:16:52.173705 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a618ae-842d-45c7-b527-0ca74b76124c admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:16:52.185991 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 899/1791] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Aug 24 06:16:51 2022] GET /identity/v3/roles => generated 1420 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:16:52.191021 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680405dc-76b5-4cb1-931a-8e36fe8b076d admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:16:52.209263 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680405dc-76b5-4cb1-931a-8e36fe8b076d admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:16:52.221120 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe6d829-e4f4-4968-84a5-a8def2f7355e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:16:52.222022 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe6d829-e4f4-4968-84a5-a8def2f7355e admin admin] REQUEST_METHOD: `PUT` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:52.222149 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe6d829-e4f4-4968-84a5-a8def2f7355e admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:52.222390 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe6d829-e4f4-4968-84a5-a8def2f7355e admin admin] PATH_INFO: `/v3/projects/5053dda49bb14b28b9357947315170b6/users/54a17a48f603457995acbec718b6a3b7/roles/49e47cd157d743f6905f1db07ec878a7` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:52.224850 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe6d829-e4f4-4968-84a5-a8def2f7355e admin admin] RBAC: Authorizing `identity:create_grant(project_id=5053dda49bb14b28b9357947315170b6, user_id=54a17a48f603457995acbec718b6a3b7, role_id=49e47cd157d743f6905f1db07ec878a7)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:16:52.230988 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe6d829-e4f4-4968-84a5-a8def2f7355e admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:16:52.261243 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 893/1792] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Aug 24 06:16:52 2022] PUT /identity/v3/projects/5053dda49bb14b28b9357947315170b6/users/54a17a48f603457995acbec718b6a3b7/roles/49e47cd157d743f6905f1db07ec878a7 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:16:52.281252 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50367d25-2ac1-431c-9c76-93079d80aeb5 None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:52.281545 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50367d25-2ac1-431c-9c76-93079d80aeb5 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:52.281820 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50367d25-2ac1-431c-9c76-93079d80aeb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:52.306996 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe6d829-e4f4-4968-84a5-a8def2f7355e admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:16:52.309863 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-ffe6d829-e4f4-4968-84a5-a8def2f7355e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:16:52.423655 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-50367d25-2ac1-431c-9c76-93079d80aeb5 None None] MFA Rules not processed for user `54a17a48f603457995acbec718b6a3b7`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:16:52.427862 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-50367d25-2ac1-431c-9c76-93079d80aeb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:16:52.624383 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe6d829-e4f4-4968-84a5-a8def2f7355e admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:16:52.628793 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 900/1793] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:16:52 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 350 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:16:52.637328 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e93356-cc23-49c6-9615-37fe4cd4788c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14d4fed04fb144d08918bd5d4f808d23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:16:52.638176 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e93356-cc23-49c6-9615-37fe4cd4788c service swift] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:52.638604 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e93356-cc23-49c6-9615-37fe4cd4788c service swift] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:52.639020 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e93356-cc23-49c6-9615-37fe4cd4788c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:52.704257 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50367d25-2ac1-431c-9c76-93079d80aeb5 None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:16:52.706562 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-50367d25-2ac1-431c-9c76-93079d80aeb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:16:52.806835 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:16:52.827152 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e93356-cc23-49c6-9615-37fe4cd4788c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:16:52.833155 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e93356-cc23-49c6-9615-37fe4cd4788c service swift] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:16:52.891232 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 894/1794] 10.222.0.62 () {60 vars in 1384 bytes} [Wed Aug 24 06:16:52 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 584 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 24 06:16:52.917426 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50367d25-2ac1-431c-9c76-93079d80aeb5 None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:16:52.928722 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56dd3a00-1996-4a5f-8de1-43331ffcd606 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c4def55ac0e48b29836bc5c64bc7402', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:16:52.929467 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dd3a00-1996-4a5f-8de1-43331ffcd606 service cinder] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:52.929895 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dd3a00-1996-4a5f-8de1-43331ffcd606 service cinder] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:52.930636 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dd3a00-1996-4a5f-8de1-43331ffcd606 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:52.958943 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56dd3a00-1996-4a5f-8de1-43331ffcd606 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:16:52.962335 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56dd3a00-1996-4a5f-8de1-43331ffcd606 service cinder] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:16:53.016774 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 901/1795] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:16:52 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 06:16:53.335070 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e93356-cc23-49c6-9615-37fe4cd4788c service swift] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:16:53.347942 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e93356-cc23-49c6-9615-37fe4cd4788c service swift] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:16:53.365052 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b5df56-a549-4d69-9ee2-4941c632fb96 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c4def55ac0e48b29836bc5c64bc7402', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:16:53.365692 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b5df56-a549-4d69-9ee2-4941c632fb96 service cinder] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:53.365973 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b5df56-a549-4d69-9ee2-4941c632fb96 service cinder] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:53.366244 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b5df56-a549-4d69-9ee2-4941c632fb96 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:53.377238 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-72b5df56-a549-4d69-9ee2-4941c632fb96 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:16:53.561645 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b5df56-a549-4d69-9ee2-4941c632fb96 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:16:53.564024 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b5df56-a549-4d69-9ee2-4941c632fb96 service cinder] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:16:53.579738 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:16:53.607496 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 895/1796] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:16:53 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 273 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:16:53.957510 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56dd3a00-1996-4a5f-8de1-43331ffcd606 service cinder] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:16:53.959035 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-56dd3a00-1996-4a5f-8de1-43331ffcd606 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:16:54.009525 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c6d910-0335-43b4-8dc3-43846d52e49a None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:54.010307 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c6d910-0335-43b4-8dc3-43846d52e49a None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:54.013440 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c6d910-0335-43b4-8dc3-43846d52e49a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:54.073470 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-f2c6d910-0335-43b4-8dc3-43846d52e49a None None] MFA Rules not processed for user `0bb6c79db75e4d169331c02133d1ec32`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:16:54.079365 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-f2c6d910-0335-43b4-8dc3-43846d52e49a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:16:54.167007 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:16:54.201471 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56dd3a00-1996-4a5f-8de1-43331ffcd606 service cinder] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:16:54.209970 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0690738d-569a-44a8-9d72-885c195394e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:16:54.211065 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0690738d-569a-44a8-9d72-885c195394e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:54.211065 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0690738d-569a-44a8-9d72-885c195394e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:54.211313 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0690738d-569a-44a8-9d72-885c195394e0 admin admin] PATH_INFO: `/v3/projects/7e2a54e00e374c1d9f11ce2269ee59fa` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:54.214019 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0690738d-569a-44a8-9d72-885c195394e0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7e2a54e00e374c1d9f11ce2269ee59fa)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:16:54.216486 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0690738d-569a-44a8-9d72-885c195394e0 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:16:54.235995 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 896/1797] 10.222.0.20 () {58 vars in 1015 bytes} [Wed Aug 24 06:16:54 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 06:16:54.249150 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c6d910-0335-43b4-8dc3-43846d52e49a None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:16:54.251424 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-f2c6d910-0335-43b4-8dc3-43846d52e49a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:16:54.308326 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c6d910-0335-43b4-8dc3-43846d52e49a None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:16:54.321688 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45036442-2a25-4f7a-979e-28f6b4fa002b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bb6c79db75e4d169331c02133d1ec32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:16:54.323679 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45036442-2a25-4f7a-979e-28f6b4fa002b service neutron] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:54.324370 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45036442-2a25-4f7a-979e-28f6b4fa002b service neutron] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:54.324776 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45036442-2a25-4f7a-979e-28f6b4fa002b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:54.336517 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45036442-2a25-4f7a-979e-28f6b4fa002b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:16:54.338805 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45036442-2a25-4f7a-979e-28f6b4fa002b service neutron] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:16:54.350414 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 897/1798] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:16:54 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:16:54.445601 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-0690738d-569a-44a8-9d72-885c195394e0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7e2a54e00e374c1d9f11ce2269ee59fa', 'request_id': 'req-0690738d-569a-44a8-9d72-885c195394e0'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:16:54.531521 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-0690738d-569a-44a8-9d72-885c195394e0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7e2a54e00e374c1d9f11ce2269ee59fa', 'request_id': 'req-0690738d-569a-44a8-9d72-885c195394e0'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:16:54.531521 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-0690738d-569a-44a8-9d72-885c195394e0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7e2a54e00e374c1d9f11ce2269ee59fa', 'request_id': 'req-0690738d-569a-44a8-9d72-885c195394e0'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:16:54.534369 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45036442-2a25-4f7a-979e-28f6b4fa002b service neutron] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:16:54.536799 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-45036442-2a25-4f7a-979e-28f6b4fa002b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:16:54.561471 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45036442-2a25-4f7a-979e-28f6b4fa002b service neutron] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:16:54.823758 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a08d2a-9a9d-4739-afad-218d36c5ef35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:16:54.823758 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a08d2a-9a9d-4739-afad-218d36c5ef35 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:54.823758 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a08d2a-9a9d-4739-afad-218d36c5ef35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:54.823758 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a08d2a-9a9d-4739-afad-218d36c5ef35 admin admin] PATH_INFO: `/v3/users/e632e4d5279c4046b2c0976449152df4` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:54.825741 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a08d2a-9a9d-4739-afad-218d36c5ef35 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e632e4d5279c4046b2c0976449152df4)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:16:54.825741 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a08d2a-9a9d-4739-afad-218d36c5ef35 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:16:55.071021 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-0690738d-569a-44a8-9d72-885c195394e0 admin admin] The token cache is being invalidate because project 7e2a54e00e374c1d9f11ce2269ee59fa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81851) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:16:55.079628 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-0690738d-569a-44a8-9d72-885c195394e0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:16:55.079628 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-0690738d-569a-44a8-9d72-885c195394e0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:16:55.079628 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 902/1799] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Aug 24 06:16:53 2022] DELETE /identity/v3/projects/7e2a54e00e374c1d9f11ce2269ee59fa => generated 0 bytes in 1117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:16:55.083282 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9582f51a-93ae-4aa0-b77d-182faa1e4ab9 None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:55.086083 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9582f51a-93ae-4aa0-b77d-182faa1e4ab9 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:55.086251 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9582f51a-93ae-4aa0-b77d-182faa1e4ab9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:55.148615 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-9582f51a-93ae-4aa0-b77d-182faa1e4ab9 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:16:55.148615 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-9582f51a-93ae-4aa0-b77d-182faa1e4ab9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:16:55.188908 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 903/1800] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:16:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:16:55.199906 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec3bb7b-541e-479f-9a5d-3c15579ca1ef None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:55.200347 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec3bb7b-541e-479f-9a5d-3c15579ca1ef None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:55.200662 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec3bb7b-541e-479f-9a5d-3c15579ca1ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:55.261895 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-dec3bb7b-541e-479f-9a5d-3c15579ca1ef None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:16:55.264703 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-dec3bb7b-541e-479f-9a5d-3c15579ca1ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:16:55.270611 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-c4a08d2a-9a9d-4739-afad-218d36c5ef35 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e632e4d5279c4046b2c0976449152df4'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:16:55.303344 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-c4a08d2a-9a9d-4739-afad-218d36c5ef35 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e632e4d5279c4046b2c0976449152df4'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:16:55.303344 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-c4a08d2a-9a9d-4739-afad-218d36c5ef35 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e632e4d5279c4046b2c0976449152df4'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:16:55.312826 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 904/1801] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:16:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:16:55.313805 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec3bb7b-541e-479f-9a5d-3c15579ca1ef None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:16:55.342951 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec3bb7b-541e-479f-9a5d-3c15579ca1ef None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:16:55.361210 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec817f38-3a50-4d70-a4db-02d8f173d6d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:16:55.362519 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec817f38-3a50-4d70-a4db-02d8f173d6d2 admin admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:55.362830 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec817f38-3a50-4d70-a4db-02d8f173d6d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:55.363076 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec817f38-3a50-4d70-a4db-02d8f173d6d2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:55.363761 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec817f38-3a50-4d70-a4db-02d8f173d6d2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:16:55.364065 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec817f38-3a50-4d70-a4db-02d8f173d6d2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:16:55.365760 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec817f38-3a50-4d70-a4db-02d8f173d6d2 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:16:55.404268 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 905/1802] 10.222.0.62 () {58 vars in 1153 bytes} [Wed Aug 24 06:16:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:16:55.407143 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec817f38-3a50-4d70-a4db-02d8f173d6d2 admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:16:55.428347 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-c4a08d2a-9a9d-4739-afad-218d36c5ef35 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e632e4d5279c4046b2c0976449152df4'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:16:55.449447 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec817f38-3a50-4d70-a4db-02d8f173d6d2 admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:16:55.459594 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-c4a08d2a-9a9d-4739-afad-218d36c5ef35 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e632e4d5279c4046b2c0976449152df4'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:16:55.478210 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 898/1803] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Aug 24 06:16:54 2022] DELETE /identity/v3/users/e632e4d5279c4046b2c0976449152df4 => generated 0 bytes in 944 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:16:55.484996 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecd79ef-3af6-4de3-b7cc-6571d7c4d6c5 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:55.485438 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecd79ef-3af6-4de3-b7cc-6571d7c4d6c5 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:55.492003 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecd79ef-3af6-4de3-b7cc-6571d7c4d6c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:55.528091 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-7ecd79ef-3af6-4de3-b7cc-6571d7c4d6c5 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:16:55.530447 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-7ecd79ef-3af6-4de3-b7cc-6571d7c4d6c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:16:55.568427 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:16:55.677071 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66eaa0f2-4be8-45a4-8c00-084acd55efe4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:16:55.677744 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66eaa0f2-4be8-45a4-8c00-084acd55efe4 admin admin] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:55.678065 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66eaa0f2-4be8-45a4-8c00-084acd55efe4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:55.678445 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66eaa0f2-4be8-45a4-8c00-084acd55efe4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:55.679256 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66eaa0f2-4be8-45a4-8c00-084acd55efe4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:16:55.685791 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66eaa0f2-4be8-45a4-8c00-084acd55efe4 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:16:55.702399 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 899/1804] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:16:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:16:55.742415 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecd79ef-3af6-4de3-b7cc-6571d7c4d6c5 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:16:55.745047 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-7ecd79ef-3af6-4de3-b7cc-6571d7c4d6c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:16:55.792361 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 906/1805] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Aug 24 06:16:55 2022] POST /identity/v3/projects => generated 385 bytes in 386 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:16:55.797570 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66eaa0f2-4be8-45a4-8c00-084acd55efe4 admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:16:55.802037 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-66eaa0f2-4be8-45a4-8c00-084acd55efe4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:16:55.848783 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:16:55.920848 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecd79ef-3af6-4de3-b7cc-6571d7c4d6c5 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:16:55.982068 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66eaa0f2-4be8-45a4-8c00-084acd55efe4 admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:16:55.996202 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e759dda7-e345-44ba-a364-4b1304788447 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:16:55.996202 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e759dda7-e345-44ba-a364-4b1304788447 admin admin] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:55.996202 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e759dda7-e345-44ba-a364-4b1304788447 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:55.996967 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e759dda7-e345-44ba-a364-4b1304788447 admin admin] PATH_INFO: `/v3/users` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:55.998023 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e759dda7-e345-44ba-a364-4b1304788447 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:16:56.001914 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e759dda7-e345-44ba-a364-4b1304788447 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:16:56.052731 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2671f1ba-6813-411b-921f-615b34d812ac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bb6c79db75e4d169331c02133d1ec32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:16:56.053766 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2671f1ba-6813-411b-921f-615b34d812ac service neutron] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:56.054196 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2671f1ba-6813-411b-921f-615b34d812ac service neutron] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:56.054647 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2671f1ba-6813-411b-921f-615b34d812ac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:56.066215 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2671f1ba-6813-411b-921f-615b34d812ac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:16:56.070287 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2671f1ba-6813-411b-921f-615b34d812ac service neutron] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:16:56.084346 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 900/1806] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:16:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 06:16:56.134465 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 907/1807] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Aug 24 06:16:55 2022] POST /identity/v3/users => generated 293 bytes in 338 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:16:56.139878 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2671f1ba-6813-411b-921f-615b34d812ac service neutron] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:16:56.155321 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2671f1ba-6813-411b-921f-615b34d812ac service neutron] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:16:56.167418 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc62511-2905-4659-ad04-c724e74888ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:16:56.169617 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc62511-2905-4659-ad04-c724e74888ee admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:56.169617 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc62511-2905-4659-ad04-c724e74888ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:56.169617 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc62511-2905-4659-ad04-c724e74888ee admin admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:56.170447 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc62511-2905-4659-ad04-c724e74888ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:16:56.172362 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc62511-2905-4659-ad04-c724e74888ee admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:16:56.192638 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 901/1808] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Aug 24 06:16:56 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 06:16:56.195258 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e759dda7-e345-44ba-a364-4b1304788447 admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:16:56.215606 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e759dda7-e345-44ba-a364-4b1304788447 admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:16:56.219117 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bb6045-1054-48fe-992a-4162c4364712 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:16:56.219760 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bb6045-1054-48fe-992a-4162c4364712 admin admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:56.220050 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bb6045-1054-48fe-992a-4162c4364712 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:56.220365 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bb6045-1054-48fe-992a-4162c4364712 admin admin] PATH_INFO: `/v3/roles` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:56.221876 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97bb6045-1054-48fe-992a-4162c4364712 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:16:56.223927 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97bb6045-1054-48fe-992a-4162c4364712 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:16:56.240046 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 908/1809] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Aug 24 06:16:56 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:16:56.245001 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc62511-2905-4659-ad04-c724e74888ee admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:16:56.258160 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc62511-2905-4659-ad04-c724e74888ee admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:16:56.273504 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2f0d61-95d2-4be2-9b68-5feff56d9bd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:16:56.274223 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2f0d61-95d2-4be2-9b68-5feff56d9bd9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:56.274550 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2f0d61-95d2-4be2-9b68-5feff56d9bd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:56.274721 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2f0d61-95d2-4be2-9b68-5feff56d9bd9 admin admin] PATH_INFO: `/v3/projects/fa97a9b99b8e448eb6e795b3f02630ab/users/a14634b2ca3446d3be99cbf082ad0cbc/roles/580aa7dbb38d4cff8f9d1bb8983f99ef` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:56.305063 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2f0d61-95d2-4be2-9b68-5feff56d9bd9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa97a9b99b8e448eb6e795b3f02630ab, user_id=a14634b2ca3446d3be99cbf082ad0cbc, role_id=580aa7dbb38d4cff8f9d1bb8983f99ef)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:16:56.308842 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2f0d61-95d2-4be2-9b68-5feff56d9bd9 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:16:56.430042 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 902/1810] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Aug 24 06:16:56 2022] PUT /identity/v3/projects/fa97a9b99b8e448eb6e795b3f02630ab/users/a14634b2ca3446d3be99cbf082ad0cbc/roles/580aa7dbb38d4cff8f9d1bb8983f99ef => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 06:16:56.440251 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c554ab-0819-4bfb-a947-c5f9552b5a6a None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:56.440869 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c554ab-0819-4bfb-a947-c5f9552b5a6a None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:56.441319 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c554ab-0819-4bfb-a947-c5f9552b5a6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:56.482403 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-f6c554ab-0819-4bfb-a947-c5f9552b5a6a None None] MFA Rules not processed for user `a14634b2ca3446d3be99cbf082ad0cbc`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:16:56.483619 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-f6c554ab-0819-4bfb-a947-c5f9552b5a6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:16:56.669116 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 909/1811] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:16:56 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:16:56.736097 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2f0d61-95d2-4be2-9b68-5feff56d9bd9 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:16:56.736308 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-5a2f0d61-95d2-4be2-9b68-5feff56d9bd9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:16:56.821177 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2f0d61-95d2-4be2-9b68-5feff56d9bd9 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:16:56.831622 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be542806-0b7c-4985-8a75-368975e8ff32 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c4def55ac0e48b29836bc5c64bc7402', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:16:56.836100 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be542806-0b7c-4985-8a75-368975e8ff32 service cinder] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:56.836913 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be542806-0b7c-4985-8a75-368975e8ff32 service cinder] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:56.841844 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be542806-0b7c-4985-8a75-368975e8ff32 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:56.863445 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be542806-0b7c-4985-8a75-368975e8ff32 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:16:56.865243 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be542806-0b7c-4985-8a75-368975e8ff32 service cinder] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:16:56.892647 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 903/1812] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:16:56 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:16:56.915618 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:16:57.151864 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:16:57.917875 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c554ab-0819-4bfb-a947-c5f9552b5a6a None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:16:57.922469 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-f6c554ab-0819-4bfb-a947-c5f9552b5a6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:16:57.942889 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:16:58.154300 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f456fba-d279-44ff-a087-cb299823e13e None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:58.155940 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f456fba-d279-44ff-a087-cb299823e13e None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:58.156331 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f456fba-d279-44ff-a087-cb299823e13e None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:58.157905 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 904/1813] 10.222.0.62 () {54 vars in 921 bytes} [Wed Aug 24 06:16:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 06:16:58.221454 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c554ab-0819-4bfb-a947-c5f9552b5a6a None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:16:58.238388 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3c5eb6-63e3-4f1d-b029-53c0d74d533c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:16:58.239113 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3c5eb6-63e3-4f1d-b029-53c0d74d533c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:58.239318 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3c5eb6-63e3-4f1d-b029-53c0d74d533c admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:58.241122 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3c5eb6-63e3-4f1d-b029-53c0d74d533c admin admin] PATH_INFO: `/v3/projects/983f56f36a63477690fe70282ae57787` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:58.244373 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3c5eb6-63e3-4f1d-b029-53c0d74d533c admin admin] RBAC: Authorizing `identity:delete_project(project_id=983f56f36a63477690fe70282ae57787)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:16:58.249041 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3c5eb6-63e3-4f1d-b029-53c0d74d533c admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:16:58.294767 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae18f42a-f759-477b-bc2d-c8db3f8ecaf6 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:58.295573 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae18f42a-f759-477b-bc2d-c8db3f8ecaf6 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:58.296128 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae18f42a-f759-477b-bc2d-c8db3f8ecaf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:58.349905 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-ae18f42a-f759-477b-bc2d-c8db3f8ecaf6 None None] MFA Rules not processed for user `fb6d3743340241f98cacb9e0d063298c`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:16:58.353389 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-ae18f42a-f759-477b-bc2d-c8db3f8ecaf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:16:58.463551 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-ac3c5eb6-63e3-4f1d-b029-53c0d74d533c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '983f56f36a63477690fe70282ae57787', 'request_id': 'req-ac3c5eb6-63e3-4f1d-b029-53c0d74d533c'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:16:58.484584 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-ac3c5eb6-63e3-4f1d-b029-53c0d74d533c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '983f56f36a63477690fe70282ae57787', 'request_id': 'req-ac3c5eb6-63e3-4f1d-b029-53c0d74d533c'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:16:58.498704 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 905/1814] 10.222.0.62 () {58 vars in 1018 bytes} [Wed Aug 24 06:16:58 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 06:16:58.499641 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-ac3c5eb6-63e3-4f1d-b029-53c0d74d533c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '983f56f36a63477690fe70282ae57787', 'request_id': 'req-ac3c5eb6-63e3-4f1d-b029-53c0d74d533c'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:16:58.581523 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:16:58.588127 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-ac3c5eb6-63e3-4f1d-b029-53c0d74d533c admin admin] The token cache is being invalidate because project 983f56f36a63477690fe70282ae57787 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81851) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:16:58.588424 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-ac3c5eb6-63e3-4f1d-b029-53c0d74d533c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:16:58.597138 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-ac3c5eb6-63e3-4f1d-b029-53c0d74d533c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:16:58.602507 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 910/1815] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Aug 24 06:16:57 2022] DELETE /identity/v3/projects/983f56f36a63477690fe70282ae57787 => generated 0 bytes in 683 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:16:58.604389 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae18f42a-f759-477b-bc2d-c8db3f8ecaf6 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:16:58.632800 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae18f42a-f759-477b-bc2d-c8db3f8ecaf6 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:16:58.747483 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e70d7a-206d-4309-b2f2-4efc55074310 None None] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:58.747994 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e70d7a-206d-4309-b2f2-4efc55074310 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:58.748438 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e70d7a-206d-4309-b2f2-4efc55074310 None None] PATH_INFO: `/` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:58.752765 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 911/1816] 10.222.0.62 () {54 vars in 921 bytes} [Wed Aug 24 06:16:58 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:16:58.876566 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a174d32-ad6b-4aef-b4e3-4d2894f8715f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:16:58.879010 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a174d32-ad6b-4aef-b4e3-4d2894f8715f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:58.879297 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a174d32-ad6b-4aef-b4e3-4d2894f8715f admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:58.879574 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a174d32-ad6b-4aef-b4e3-4d2894f8715f admin admin] PATH_INFO: `/v3/users/aa4f92933e8541ab91d6f074572fa6ee` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:58.884749 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43316d63-db8e-4ae7-b699-cc0851ef91b0 None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:58.885029 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43316d63-db8e-4ae7-b699-cc0851ef91b0 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:58.885468 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43316d63-db8e-4ae7-b699-cc0851ef91b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:58.886580 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a174d32-ad6b-4aef-b4e3-4d2894f8715f admin admin] RBAC: Authorizing `identity:delete_user(user_id=aa4f92933e8541ab91d6f074572fa6ee)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:16:58.891725 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a174d32-ad6b-4aef-b4e3-4d2894f8715f admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:16:58.930558 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-43316d63-db8e-4ae7-b699-cc0851ef91b0 None None] MFA Rules not processed for user `fb6d3743340241f98cacb9e0d063298c`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:16:58.930558 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-43316d63-db8e-4ae7-b699-cc0851ef91b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:16:58.960532 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:16:59.113177 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 912/1817] 10.222.0.62 () {58 vars in 1018 bytes} [Wed Aug 24 06:16:58 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:16:59.114187 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43316d63-db8e-4ae7-b699-cc0851ef91b0 None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:16:59.116527 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-43316d63-db8e-4ae7-b699-cc0851ef91b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:16:59.138398 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43316d63-db8e-4ae7-b699-cc0851ef91b0 None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:16:59.154376 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb19f432-c781-4792-8849-3209702419e5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb6d3743340241f98cacb9e0d063298c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:16:59.155596 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb19f432-c781-4792-8849-3209702419e5 service nova] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:59.159532 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb19f432-c781-4792-8849-3209702419e5 service nova] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:59.160869 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb19f432-c781-4792-8849-3209702419e5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:59.194189 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-fb19f432-c781-4792-8849-3209702419e5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:16:59.242124 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb19f432-c781-4792-8849-3209702419e5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:16:59.244171 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb19f432-c781-4792-8849-3209702419e5 service nova] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:16:59.282938 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 913/1818] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:16:59 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:16:59.298549 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-3a174d32-ad6b-4aef-b4e3-4d2894f8715f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aa4f92933e8541ab91d6f074572fa6ee'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:16:59.307949 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-3a174d32-ad6b-4aef-b4e3-4d2894f8715f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aa4f92933e8541ab91d6f074572fa6ee'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:16:59.313510 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-3a174d32-ad6b-4aef-b4e3-4d2894f8715f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aa4f92933e8541ab91d6f074572fa6ee'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:16:59.373545 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-3a174d32-ad6b-4aef-b4e3-4d2894f8715f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aa4f92933e8541ab91d6f074572fa6ee'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:16:59.412310 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-3a174d32-ad6b-4aef-b4e3-4d2894f8715f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aa4f92933e8541ab91d6f074572fa6ee'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:16:59.460404 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 906/1819] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Aug 24 06:16:58 2022] DELETE /identity/v3/users/aa4f92933e8541ab91d6f074572fa6ee => generated 0 bytes in 852 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:16:59.461422 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb19f432-c781-4792-8849-3209702419e5 service nova] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:16:59.472779 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:16:59.478076 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb19f432-c781-4792-8849-3209702419e5 service nova] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:16:59.651326 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702418c4-b0f2-4ee8-b7ba-3b2a2024b0b0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb6d3743340241f98cacb9e0d063298c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:16:59.652101 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702418c4-b0f2-4ee8-b7ba-3b2a2024b0b0 service nova] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:16:59.653844 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702418c4-b0f2-4ee8-b7ba-3b2a2024b0b0 service nova] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:16:59.654264 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702418c4-b0f2-4ee8-b7ba-3b2a2024b0b0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:16:59.659033 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-702418c4-b0f2-4ee8-b7ba-3b2a2024b0b0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:16:59.734344 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702418c4-b0f2-4ee8-b7ba-3b2a2024b0b0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:16:59.744245 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702418c4-b0f2-4ee8-b7ba-3b2a2024b0b0 service nova] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:16:59.790202 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 914/1820] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:16:59 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 06:17:00.248070 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:17:00.679064 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a174d32-ad6b-4aef-b4e3-4d2894f8715f admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:17:00.680557 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-3a174d32-ad6b-4aef-b4e3-4d2894f8715f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:17:00.705224 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702418c4-b0f2-4ee8-b7ba-3b2a2024b0b0 service nova] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:17:00.707711 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-702418c4-b0f2-4ee8-b7ba-3b2a2024b0b0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:17:00.762291 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a174d32-ad6b-4aef-b4e3-4d2894f8715f admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:17:00.767355 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec2491f-efc6-4433-b6e1-652b76bdf818 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:17:00.768448 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec2491f-efc6-4433-b6e1-652b76bdf818 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:00.768898 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec2491f-efc6-4433-b6e1-652b76bdf818 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:00.769448 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec2491f-efc6-4433-b6e1-652b76bdf818 admin admin] PATH_INFO: `/v3/users/e3b7ebadfed24f77895403aa56e4ecaa` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:00.771035 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec2491f-efc6-4433-b6e1-652b76bdf818 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e3b7ebadfed24f77895403aa56e4ecaa)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:17:00.778334 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec2491f-efc6-4433-b6e1-652b76bdf818 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:17:00.817738 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702418c4-b0f2-4ee8-b7ba-3b2a2024b0b0 service nova] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:17:00.822895 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9216dd65-7e08-4073-9433-8f2d36e7b799 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:17:00.823533 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9216dd65-7e08-4073-9433-8f2d36e7b799 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:00.823811 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9216dd65-7e08-4073-9433-8f2d36e7b799 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:00.824080 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9216dd65-7e08-4073-9433-8f2d36e7b799 admin admin] PATH_INFO: `/v3/projects/89ed44a904d84eceae80b44443ae6413` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:00.825218 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9216dd65-7e08-4073-9433-8f2d36e7b799 admin admin] RBAC: Authorizing `identity:delete_project(project_id=89ed44a904d84eceae80b44443ae6413)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:17:00.827112 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9216dd65-7e08-4073-9433-8f2d36e7b799 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:17:01.007783 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-9216dd65-7e08-4073-9433-8f2d36e7b799 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '89ed44a904d84eceae80b44443ae6413', 'request_id': 'req-9216dd65-7e08-4073-9433-8f2d36e7b799'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:17:01.008535 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-7ec2491f-efc6-4433-b6e1-652b76bdf818 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e3b7ebadfed24f77895403aa56e4ecaa'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:17:01.013450 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-9216dd65-7e08-4073-9433-8f2d36e7b799 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '89ed44a904d84eceae80b44443ae6413', 'request_id': 'req-9216dd65-7e08-4073-9433-8f2d36e7b799'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:17:01.015624 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-7ec2491f-efc6-4433-b6e1-652b76bdf818 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e3b7ebadfed24f77895403aa56e4ecaa'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:17:01.016317 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-7ec2491f-efc6-4433-b6e1-652b76bdf818 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e3b7ebadfed24f77895403aa56e4ecaa'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:17:01.019606 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-9216dd65-7e08-4073-9433-8f2d36e7b799 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '89ed44a904d84eceae80b44443ae6413', 'request_id': 'req-9216dd65-7e08-4073-9433-8f2d36e7b799'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:17:01.104178 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-7ec2491f-efc6-4433-b6e1-652b76bdf818 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e3b7ebadfed24f77895403aa56e4ecaa'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:17:01.117154 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-7ec2491f-efc6-4433-b6e1-652b76bdf818 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e3b7ebadfed24f77895403aa56e4ecaa'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:17:01.123186 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 907/1821] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Aug 24 06:17:00 2022] DELETE /identity/v3/users/e3b7ebadfed24f77895403aa56e4ecaa => generated 0 bytes in 445 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:17:01.127575 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5aded8-ca7e-4890-aa35-e7c61f83e05c None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:01.127869 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5aded8-ca7e-4890-aa35-e7c61f83e05c None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:01.128146 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5aded8-ca7e-4890-aa35-e7c61f83e05c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:01.150611 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-ea5aded8-ca7e-4890-aa35-e7c61f83e05c None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:17:01.153552 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-ea5aded8-ca7e-4890-aa35-e7c61f83e05c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:17:01.200954 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-9216dd65-7e08-4073-9433-8f2d36e7b799 admin admin] The token cache is being invalidate because project 89ed44a904d84eceae80b44443ae6413 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81851) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:17:01.201404 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-9216dd65-7e08-4073-9433-8f2d36e7b799 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:17:01.202173 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-9216dd65-7e08-4073-9433-8f2d36e7b799 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:17:01.212516 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 915/1822] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Aug 24 06:17:00 2022] DELETE /identity/v3/projects/89ed44a904d84eceae80b44443ae6413 => generated 0 bytes in 502 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:17:01.212516 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1e359c-02a1-4ee0-8334-94468d62c607 None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:01.213021 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1e359c-02a1-4ee0-8334-94468d62c607 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:01.213560 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1e359c-02a1-4ee0-8334-94468d62c607 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:01.227885 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 908/1823] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:17:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 06:17:01.242224 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-2e1e359c-02a1-4ee0-8334-94468d62c607 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:17:01.243758 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-2e1e359c-02a1-4ee0-8334-94468d62c607 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:17:01.251947 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5aded8-ca7e-4890-aa35-e7c61f83e05c None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:17:01.252074 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 916/1824] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:17:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:17:01.266395 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-ea5aded8-ca7e-4890-aa35-e7c61f83e05c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:17:01.269955 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7334da0-89f7-410c-8933-6d98661508b5 None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:01.270380 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7334da0-89f7-410c-8933-6d98661508b5 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:01.270760 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7334da0-89f7-410c-8933-6d98661508b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:01.295897 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-f7334da0-89f7-410c-8933-6d98661508b5 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:17:01.295897 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-f7334da0-89f7-410c-8933-6d98661508b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:17:01.307582 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 917/1825] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:17:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 06:17:01.308896 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7334da0-89f7-410c-8933-6d98661508b5 None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:17:01.322255 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5aded8-ca7e-4890-aa35-e7c61f83e05c None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:17:01.326715 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386d5dd0-fae6-442b-9637-c721ddf083f1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bb6c79db75e4d169331c02133d1ec32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:17:01.327325 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386d5dd0-fae6-442b-9637-c721ddf083f1 service neutron] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:01.327600 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386d5dd0-fae6-442b-9637-c721ddf083f1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:01.329880 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386d5dd0-fae6-442b-9637-c721ddf083f1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:01.338597 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386d5dd0-fae6-442b-9637-c721ddf083f1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:17:01.340182 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386d5dd0-fae6-442b-9637-c721ddf083f1 service neutron] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:17:01.340863 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7334da0-89f7-410c-8933-6d98661508b5 None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:17:01.346204 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3673395b-f929-4a07-b2e2-2c6589e93b7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:17:01.346740 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3673395b-f929-4a07-b2e2-2c6589e93b7e admin admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:01.347043 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3673395b-f929-4a07-b2e2-2c6589e93b7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:01.350318 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3673395b-f929-4a07-b2e2-2c6589e93b7e admin admin] PATH_INFO: `/v3/domains` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:01.350318 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3673395b-f929-4a07-b2e2-2c6589e93b7e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:17:01.350318 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3673395b-f929-4a07-b2e2-2c6589e93b7e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:17:01.351077 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3673395b-f929-4a07-b2e2-2c6589e93b7e admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:17:01.354870 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 909/1826] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:17:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 06:17:01.367224 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 918/1827] 10.222.0.62 () {58 vars in 1153 bytes} [Wed Aug 24 06:17:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:17:01.371532 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386d5dd0-fae6-442b-9637-c721ddf083f1 service neutron] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:17:01.385925 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386d5dd0-fae6-442b-9637-c721ddf083f1 service neutron] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:17:01.392150 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d3c1af-780f-4f7b-9c09-cb4d8cc8d720 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:17:01.392818 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d3c1af-780f-4f7b-9c09-cb4d8cc8d720 admin admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:01.396287 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d3c1af-780f-4f7b-9c09-cb4d8cc8d720 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:01.396679 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d3c1af-780f-4f7b-9c09-cb4d8cc8d720 admin admin] PATH_INFO: `/v3/projects` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:01.397278 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d3c1af-780f-4f7b-9c09-cb4d8cc8d720 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:17:01.399349 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d3c1af-780f-4f7b-9c09-cb4d8cc8d720 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:17:01.425642 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:17:01.542144 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 910/1828] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Aug 24 06:17:01 2022] POST /identity/v3/projects => generated 389 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:17:01.542607 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3673395b-f929-4a07-b2e2-2c6589e93b7e admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:17:01.550738 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3673395b-f929-4a07-b2e2-2c6589e93b7e admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:17:01.617600 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6d6da2-7927-408e-b56f-b4f5d3ac9568 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:17:01.618250 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6d6da2-7927-408e-b56f-b4f5d3ac9568 admin admin] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:01.618529 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6d6da2-7927-408e-b56f-b4f5d3ac9568 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:01.618798 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6d6da2-7927-408e-b56f-b4f5d3ac9568 admin admin] PATH_INFO: `/v3/users` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:01.619510 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6d6da2-7927-408e-b56f-b4f5d3ac9568 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:17:01.621469 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6d6da2-7927-408e-b56f-b4f5d3ac9568 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:17:01.727327 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 919/1829] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Aug 24 06:17:01 2022] POST /identity/v3/users => generated 295 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 06:17:01.730954 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d3c1af-780f-4f7b-9c09-cb4d8cc8d720 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:17:01.745655 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d3c1af-780f-4f7b-9c09-cb4d8cc8d720 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:17:01.750547 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86c23ed-d66a-4c00-a459-de30f89280b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:17:01.751279 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86c23ed-d66a-4c00-a459-de30f89280b9 admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:01.751859 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86c23ed-d66a-4c00-a459-de30f89280b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:01.752169 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86c23ed-d66a-4c00-a459-de30f89280b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:01.753008 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86c23ed-d66a-4c00-a459-de30f89280b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:17:01.755437 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86c23ed-d66a-4c00-a459-de30f89280b9 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:17:01.765372 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 911/1830] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Aug 24 06:17:01 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:17:01.769943 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6d6da2-7927-408e-b56f-b4f5d3ac9568 admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:17:01.782419 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6d6da2-7927-408e-b56f-b4f5d3ac9568 admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:17:01.787661 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed47e4d4-62a7-45d0-9958-991e6edcec8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:17:01.788590 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed47e4d4-62a7-45d0-9958-991e6edcec8b admin admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:01.788976 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed47e4d4-62a7-45d0-9958-991e6edcec8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:01.789330 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed47e4d4-62a7-45d0-9958-991e6edcec8b admin admin] PATH_INFO: `/v3/roles` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:01.790059 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed47e4d4-62a7-45d0-9958-991e6edcec8b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:17:01.791737 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed47e4d4-62a7-45d0-9958-991e6edcec8b admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:17:01.803526 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 920/1831] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Aug 24 06:17:01 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:17:01.807420 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86c23ed-d66a-4c00-a459-de30f89280b9 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:17:01.819395 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86c23ed-d66a-4c00-a459-de30f89280b9 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:17:01.824065 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fddbc53-4f56-4f51-9e5d-f35291c559d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:17:01.824764 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fddbc53-4f56-4f51-9e5d-f35291c559d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:01.825043 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fddbc53-4f56-4f51-9e5d-f35291c559d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:01.825312 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fddbc53-4f56-4f51-9e5d-f35291c559d3 admin admin] PATH_INFO: `/v3/projects/06831a6993814effa495a95dd0bdca0e/users/cc1ba5751e5441e2a8b58d0c5a84acd9/roles/580aa7dbb38d4cff8f9d1bb8983f99ef` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:01.841603 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fddbc53-4f56-4f51-9e5d-f35291c559d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=06831a6993814effa495a95dd0bdca0e, user_id=cc1ba5751e5441e2a8b58d0c5a84acd9, role_id=580aa7dbb38d4cff8f9d1bb8983f99ef)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:17:01.844310 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fddbc53-4f56-4f51-9e5d-f35291c559d3 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:17:01.935088 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed47e4d4-62a7-45d0-9958-991e6edcec8b admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:17:01.937595 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-ed47e4d4-62a7-45d0-9958-991e6edcec8b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:17:01.942049 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 912/1832] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Aug 24 06:17:01 2022] PUT /identity/v3/projects/06831a6993814effa495a95dd0bdca0e/users/cc1ba5751e5441e2a8b58d0c5a84acd9/roles/580aa7dbb38d4cff8f9d1bb8983f99ef => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:17:01.951976 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11ea8ab-c429-4bb9-9c89-d0e3e6fe4260 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:01.952256 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11ea8ab-c429-4bb9-9c89-d0e3e6fe4260 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:01.952642 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11ea8ab-c429-4bb9-9c89-d0e3e6fe4260 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:01.954720 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed47e4d4-62a7-45d0-9958-991e6edcec8b admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:17:01.981519 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-f11ea8ab-c429-4bb9-9c89-d0e3e6fe4260 None None] MFA Rules not processed for user `cc1ba5751e5441e2a8b58d0c5a84acd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:17:01.983317 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-f11ea8ab-c429-4bb9-9c89-d0e3e6fe4260 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:17:02.016334 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c567dd2-d3c8-4f74-8cf3-6b38ab06b297 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:17:02.017042 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c567dd2-d3c8-4f74-8cf3-6b38ab06b297 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:02.017329 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c567dd2-d3c8-4f74-8cf3-6b38ab06b297 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:02.017601 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c567dd2-d3c8-4f74-8cf3-6b38ab06b297 admin admin] PATH_INFO: `/v3/projects/32898113fdce4e8987562cc754c585d5` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:02.018852 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c567dd2-d3c8-4f74-8cf3-6b38ab06b297 admin admin] RBAC: Authorizing `identity:delete_project(project_id=32898113fdce4e8987562cc754c585d5)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:17:02.021434 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c567dd2-d3c8-4f74-8cf3-6b38ab06b297 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:17:02.058362 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 913/1833] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:17:01 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 06:17:02.062795 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11ea8ab-c429-4bb9-9c89-d0e3e6fe4260 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:17:02.073571 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11ea8ab-c429-4bb9-9c89-d0e3e6fe4260 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:17:02.077951 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6669d98-72cc-47fe-a6aa-61a2beb8e0a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:17:02.078865 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6669d98-72cc-47fe-a6aa-61a2beb8e0a5 admin admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:02.079176 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6669d98-72cc-47fe-a6aa-61a2beb8e0a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:02.080680 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6669d98-72cc-47fe-a6aa-61a2beb8e0a5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:02.080680 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6669d98-72cc-47fe-a6aa-61a2beb8e0a5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:17:02.082010 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6669d98-72cc-47fe-a6aa-61a2beb8e0a5 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:17:02.144838 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-5c567dd2-d3c8-4f74-8cf3-6b38ab06b297 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '32898113fdce4e8987562cc754c585d5', 'request_id': 'req-5c567dd2-d3c8-4f74-8cf3-6b38ab06b297'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:17:02.150283 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-5c567dd2-d3c8-4f74-8cf3-6b38ab06b297 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '32898113fdce4e8987562cc754c585d5', 'request_id': 'req-5c567dd2-d3c8-4f74-8cf3-6b38ab06b297'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:17:02.156462 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-5c567dd2-d3c8-4f74-8cf3-6b38ab06b297 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '32898113fdce4e8987562cc754c585d5', 'request_id': 'req-5c567dd2-d3c8-4f74-8cf3-6b38ab06b297'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:17:02.162586 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 914/1834] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Aug 24 06:17:02 2022] POST /identity/v3/projects => generated 389 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 06:17:02.166616 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6669d98-72cc-47fe-a6aa-61a2beb8e0a5 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:17:02.228887 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6669d98-72cc-47fe-a6aa-61a2beb8e0a5 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:17:02.233937 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f21472e-bf13-4efb-9339-535f8c907007 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:17:02.234717 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f21472e-bf13-4efb-9339-535f8c907007 admin admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:02.235041 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f21472e-bf13-4efb-9339-535f8c907007 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:02.235341 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f21472e-bf13-4efb-9339-535f8c907007 admin admin] PATH_INFO: `/v3/users` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:02.236059 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f21472e-bf13-4efb-9339-535f8c907007 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:17:02.238721 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f21472e-bf13-4efb-9339-535f8c907007 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:17:02.242648 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-5c567dd2-d3c8-4f74-8cf3-6b38ab06b297 admin admin] The token cache is being invalidate because project 32898113fdce4e8987562cc754c585d5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81851) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:17:02.243035 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-5c567dd2-d3c8-4f74-8cf3-6b38ab06b297 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:17:02.243787 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-5c567dd2-d3c8-4f74-8cf3-6b38ab06b297 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:17:02.246014 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 921/1835] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Aug 24 06:17:01 2022] DELETE /identity/v3/projects/32898113fdce4e8987562cc754c585d5 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:17:02.254733 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c567dd2-d3c8-4f74-8cf3-6b38ab06b297 admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:17:02.256810 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-5c567dd2-d3c8-4f74-8cf3-6b38ab06b297 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:17:02.324979 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c567dd2-d3c8-4f74-8cf3-6b38ab06b297 admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:17:02.330058 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6df4180-d79d-480e-8140-74d76289048e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:17:02.330759 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6df4180-d79d-480e-8140-74d76289048e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:02.330986 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6df4180-d79d-480e-8140-74d76289048e admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:02.331430 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6df4180-d79d-480e-8140-74d76289048e admin admin] PATH_INFO: `/v3/users/385444b8b35a4ad2a3617d6b9f24878a` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:02.341460 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6df4180-d79d-480e-8140-74d76289048e admin admin] RBAC: Authorizing `identity:delete_user(user_id=385444b8b35a4ad2a3617d6b9f24878a)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:17:02.344005 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6df4180-d79d-480e-8140-74d76289048e admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:17:02.348291 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 915/1836] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Aug 24 06:17:02 2022] POST /identity/v3/users => generated 301 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 06:17:02.352227 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f21472e-bf13-4efb-9339-535f8c907007 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:17:02.354216 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-1f21472e-bf13-4efb-9339-535f8c907007 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:17:02.367834 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f21472e-bf13-4efb-9339-535f8c907007 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:17:02.373048 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfeb640d-8e9c-4db6-b482-f292f871ff17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:17:02.373670 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfeb640d-8e9c-4db6-b482-f292f871ff17 admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:02.374044 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfeb640d-8e9c-4db6-b482-f292f871ff17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:02.374239 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfeb640d-8e9c-4db6-b482-f292f871ff17 admin admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:02.374894 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfeb640d-8e9c-4db6-b482-f292f871ff17 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:17:02.376710 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfeb640d-8e9c-4db6-b482-f292f871ff17 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:17:02.396113 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 916/1837] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Aug 24 06:17:02 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 06:17:02.397332 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfeb640d-8e9c-4db6-b482-f292f871ff17 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:17:02.407528 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfeb640d-8e9c-4db6-b482-f292f871ff17 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:17:02.413729 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8a770b-fed9-441b-807c-70c3b79a6c69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:17:02.415990 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8a770b-fed9-441b-807c-70c3b79a6c69 admin admin] REQUEST_METHOD: `PUT` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:02.415990 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8a770b-fed9-441b-807c-70c3b79a6c69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:02.415990 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8a770b-fed9-441b-807c-70c3b79a6c69 admin admin] PATH_INFO: `/v3/projects/70e9b8312ab6415296e995d7510dc832/users/4df37ae51a5e4d9ab3309e053dbccdab/roles/49e47cd157d743f6905f1db07ec878a7` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:02.435186 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8a770b-fed9-441b-807c-70c3b79a6c69 admin admin] RBAC: Authorizing `identity:create_grant(project_id=70e9b8312ab6415296e995d7510dc832, user_id=4df37ae51a5e4d9ab3309e053dbccdab, role_id=49e47cd157d743f6905f1db07ec878a7)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:17:02.437979 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8a770b-fed9-441b-807c-70c3b79a6c69 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:17:02.555466 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 917/1838] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Aug 24 06:17:02 2022] PUT /identity/v3/projects/70e9b8312ab6415296e995d7510dc832/users/4df37ae51a5e4d9ab3309e053dbccdab/roles/49e47cd157d743f6905f1db07ec878a7 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:17:02.555466 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8a770b-fed9-441b-807c-70c3b79a6c69 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:17:02.606167 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-c6df4180-d79d-480e-8140-74d76289048e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '385444b8b35a4ad2a3617d6b9f24878a'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:17:02.623660 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-c6df4180-d79d-480e-8140-74d76289048e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '385444b8b35a4ad2a3617d6b9f24878a'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:17:02.624594 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8a770b-fed9-441b-807c-70c3b79a6c69 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:17:02.625062 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-c6df4180-d79d-480e-8140-74d76289048e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '385444b8b35a4ad2a3617d6b9f24878a'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:17:02.625707 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-c6df4180-d79d-480e-8140-74d76289048e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '385444b8b35a4ad2a3617d6b9f24878a'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:17:02.634884 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd816c6-d99c-45fc-90a4-06e4732adb94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:17:02.635528 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd816c6-d99c-45fc-90a4-06e4732adb94 admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:02.641626 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd816c6-d99c-45fc-90a4-06e4732adb94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:02.641626 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd816c6-d99c-45fc-90a4-06e4732adb94 admin admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:02.641626 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd816c6-d99c-45fc-90a4-06e4732adb94 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:17:02.641626 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd816c6-d99c-45fc-90a4-06e4732adb94 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:17:02.650982 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 918/1839] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Aug 24 06:17:02 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 06:17:02.655409 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd816c6-d99c-45fc-90a4-06e4732adb94 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:17:02.657119 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-fbd816c6-d99c-45fc-90a4-06e4732adb94 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:17:02.672606 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd816c6-d99c-45fc-90a4-06e4732adb94 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:17:02.685008 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1070ec7-0493-4114-8985-3cbb8fe098e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:17:02.685008 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1070ec7-0493-4114-8985-3cbb8fe098e3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:02.685008 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1070ec7-0493-4114-8985-3cbb8fe098e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:02.685008 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1070ec7-0493-4114-8985-3cbb8fe098e3 admin admin] PATH_INFO: `/v3/projects/70e9b8312ab6415296e995d7510dc832/users/4df37ae51a5e4d9ab3309e053dbccdab/roles/49e47cd157d743f6905f1db07ec878a7` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:02.686636 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1070ec7-0493-4114-8985-3cbb8fe098e3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=70e9b8312ab6415296e995d7510dc832, user_id=4df37ae51a5e4d9ab3309e053dbccdab, role_id=49e47cd157d743f6905f1db07ec878a7)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:17:02.693204 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1070ec7-0493-4114-8985-3cbb8fe098e3 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:17:02.706495 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 919/1840] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Aug 24 06:17:02 2022] PUT /identity/v3/projects/70e9b8312ab6415296e995d7510dc832/users/4df37ae51a5e4d9ab3309e053dbccdab/roles/49e47cd157d743f6905f1db07ec878a7 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 06:17:02.718069 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f2c6c8-0a3b-48ed-9e0f-47dad9a3112c None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:02.718616 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f2c6c8-0a3b-48ed-9e0f-47dad9a3112c None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:02.718899 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f2c6c8-0a3b-48ed-9e0f-47dad9a3112c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:02.721409 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-c6df4180-d79d-480e-8140-74d76289048e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '385444b8b35a4ad2a3617d6b9f24878a'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:17:02.751275 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 922/1841] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Aug 24 06:17:02 2022] DELETE /identity/v3/users/385444b8b35a4ad2a3617d6b9f24878a => generated 0 bytes in 501 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 06:17:02.783205 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-48f2c6c8-0a3b-48ed-9e0f-47dad9a3112c None None] MFA Rules not processed for user `4df37ae51a5e4d9ab3309e053dbccdab`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:17:02.785672 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-48f2c6c8-0a3b-48ed-9e0f-47dad9a3112c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:17:02.789593 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:17:02.875830 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:17:02.900090 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 920/1842] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:17:02 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:17:02.912884 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6df4180-d79d-480e-8140-74d76289048e admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:17:02.914371 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-c6df4180-d79d-480e-8140-74d76289048e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:17:03.031653 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6df4180-d79d-480e-8140-74d76289048e admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:17:03.042635 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b20c2cd-f758-4ef0-aac3-c30679e220ca service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c4def55ac0e48b29836bc5c64bc7402', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:17:03.043287 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b20c2cd-f758-4ef0-aac3-c30679e220ca service cinder] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:03.043554 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b20c2cd-f758-4ef0-aac3-c30679e220ca service cinder] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:03.044405 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b20c2cd-f758-4ef0-aac3-c30679e220ca service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:03.055470 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b20c2cd-f758-4ef0-aac3-c30679e220ca service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:17:03.057202 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b20c2cd-f758-4ef0-aac3-c30679e220ca service cinder] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:17:03.068034 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 923/1843] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:17:02 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:17:03.326623 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f2c6c8-0a3b-48ed-9e0f-47dad9a3112c None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:17:03.438469 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f2c6c8-0a3b-48ed-9e0f-47dad9a3112c None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:17:03.452760 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9b14a0-1811-4cdf-bfbb-6cfe9c109768 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:17:03.453425 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9b14a0-1811-4cdf-bfbb-6cfe9c109768 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:03.453708 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9b14a0-1811-4cdf-bfbb-6cfe9c109768 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:03.453983 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9b14a0-1811-4cdf-bfbb-6cfe9c109768 admin admin] PATH_INFO: `/v3/users/cc1ba5751e5441e2a8b58d0c5a84acd9` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:03.460910 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9b14a0-1811-4cdf-bfbb-6cfe9c109768 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cc1ba5751e5441e2a8b58d0c5a84acd9)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:17:03.469462 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9b14a0-1811-4cdf-bfbb-6cfe9c109768 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:17:03.549085 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b20c2cd-f758-4ef0-aac3-c30679e220ca service cinder] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:17:03.549085 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-8b20c2cd-f758-4ef0-aac3-c30679e220ca service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:17:03.563474 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b20c2cd-f758-4ef0-aac3-c30679e220ca service cinder] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:17:03.571985 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b11f4d-a7bc-4e19-9edc-a68012dcadd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:17:03.572964 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b11f4d-a7bc-4e19-9edc-a68012dcadd3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:03.575878 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b11f4d-a7bc-4e19-9edc-a68012dcadd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:03.575878 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b11f4d-a7bc-4e19-9edc-a68012dcadd3 admin admin] PATH_INFO: `/v3/projects/0c35d1486ae945e984ce36a962b1e523` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:03.575878 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b11f4d-a7bc-4e19-9edc-a68012dcadd3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0c35d1486ae945e984ce36a962b1e523)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:17:03.578018 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b11f4d-a7bc-4e19-9edc-a68012dcadd3 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:17:03.809528 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-6b9b14a0-1811-4cdf-bfbb-6cfe9c109768 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cc1ba5751e5441e2a8b58d0c5a84acd9'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:17:03.827402 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-6b9b14a0-1811-4cdf-bfbb-6cfe9c109768 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cc1ba5751e5441e2a8b58d0c5a84acd9'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:17:03.829539 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-6b9b14a0-1811-4cdf-bfbb-6cfe9c109768 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cc1ba5751e5441e2a8b58d0c5a84acd9'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:17:03.859636 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-73b11f4d-a7bc-4e19-9edc-a68012dcadd3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0c35d1486ae945e984ce36a962b1e523', 'request_id': 'req-73b11f4d-a7bc-4e19-9edc-a68012dcadd3'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:17:03.877126 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-73b11f4d-a7bc-4e19-9edc-a68012dcadd3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0c35d1486ae945e984ce36a962b1e523', 'request_id': 'req-73b11f4d-a7bc-4e19-9edc-a68012dcadd3'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:17:03.892226 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-73b11f4d-a7bc-4e19-9edc-a68012dcadd3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0c35d1486ae945e984ce36a962b1e523', 'request_id': 'req-73b11f4d-a7bc-4e19-9edc-a68012dcadd3'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:17:03.921787 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-6b9b14a0-1811-4cdf-bfbb-6cfe9c109768 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cc1ba5751e5441e2a8b58d0c5a84acd9'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:17:03.940470 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-6b9b14a0-1811-4cdf-bfbb-6cfe9c109768 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cc1ba5751e5441e2a8b58d0c5a84acd9'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:17:03.948860 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 921/1844] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Aug 24 06:17:03 2022] DELETE /identity/v3/users/cc1ba5751e5441e2a8b58d0c5a84acd9 => generated 0 bytes in 623 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:17:03.965617 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a23729-9235-4c73-ad0c-8924b637e3d3 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:03.965902 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a23729-9235-4c73-ad0c-8924b637e3d3 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:03.966480 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a23729-9235-4c73-ad0c-8924b637e3d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:03.997584 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-55a23729-9235-4c73-ad0c-8924b637e3d3 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:17:04.004637 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-55a23729-9235-4c73-ad0c-8924b637e3d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:17:04.035285 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-73b11f4d-a7bc-4e19-9edc-a68012dcadd3 admin admin] The token cache is being invalidate because project 0c35d1486ae945e984ce36a962b1e523 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81851) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:17:04.035812 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-73b11f4d-a7bc-4e19-9edc-a68012dcadd3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:17:04.038386 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-73b11f4d-a7bc-4e19-9edc-a68012dcadd3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:17:04.046515 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 924/1845] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Aug 24 06:17:03 2022] DELETE /identity/v3/projects/0c35d1486ae945e984ce36a962b1e523 => generated 0 bytes in 518 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:17:04.056072 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6d7401-5a6e-4a57-937f-4fea442ffda7 None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:04.059399 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6d7401-5a6e-4a57-937f-4fea442ffda7 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:04.060620 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6d7401-5a6e-4a57-937f-4fea442ffda7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:04.114476 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-fe6d7401-5a6e-4a57-937f-4fea442ffda7 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:17:04.116250 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-fe6d7401-5a6e-4a57-937f-4fea442ffda7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:17:04.245468 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 922/1846] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:17:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 278 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:17:04.245468 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a23729-9235-4c73-ad0c-8924b637e3d3 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:17:04.251369 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-55a23729-9235-4c73-ad0c-8924b637e3d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:17:04.283541 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a23729-9235-4c73-ad0c-8924b637e3d3 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:17:04.306465 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4ad291-b22a-4bce-978d-b601f6318728 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:17:04.307144 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4ad291-b22a-4bce-978d-b601f6318728 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:04.308762 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4ad291-b22a-4bce-978d-b601f6318728 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:04.309145 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4ad291-b22a-4bce-978d-b601f6318728 admin admin] PATH_INFO: `/v3/users/a14634b2ca3446d3be99cbf082ad0cbc` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:04.311893 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 925/1847] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:17:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:17:04.321015 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4ad291-b22a-4bce-978d-b601f6318728 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a14634b2ca3446d3be99cbf082ad0cbc)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:17:04.326314 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7779bac6-ee62-48cc-b8f8-61bc9b2c11fc None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:04.327531 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7779bac6-ee62-48cc-b8f8-61bc9b2c11fc None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:04.327531 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7779bac6-ee62-48cc-b8f8-61bc9b2c11fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:04.340801 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4ad291-b22a-4bce-978d-b601f6318728 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:17:04.416666 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-7779bac6-ee62-48cc-b8f8-61bc9b2c11fc None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:17:04.419947 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-7779bac6-ee62-48cc-b8f8-61bc9b2c11fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:17:04.438470 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 926/1848] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:17:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:17:04.442601 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7779bac6-ee62-48cc-b8f8-61bc9b2c11fc None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:17:04.443094 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-7779bac6-ee62-48cc-b8f8-61bc9b2c11fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:17:04.570731 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-5a4ad291-b22a-4bce-978d-b601f6318728 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a14634b2ca3446d3be99cbf082ad0cbc'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:17:04.591935 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-5a4ad291-b22a-4bce-978d-b601f6318728 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a14634b2ca3446d3be99cbf082ad0cbc'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:17:04.596943 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-5a4ad291-b22a-4bce-978d-b601f6318728 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a14634b2ca3446d3be99cbf082ad0cbc'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:17:04.614734 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7779bac6-ee62-48cc-b8f8-61bc9b2c11fc None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:17:04.619478 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8354bb-3c63-43c4-9a59-273555f2ee28 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bb6c79db75e4d169331c02133d1ec32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:17:04.622662 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8354bb-3c63-43c4-9a59-273555f2ee28 service neutron] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:04.622662 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8354bb-3c63-43c4-9a59-273555f2ee28 service neutron] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:04.622662 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8354bb-3c63-43c4-9a59-273555f2ee28 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:04.638163 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8354bb-3c63-43c4-9a59-273555f2ee28 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:17:04.649825 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8354bb-3c63-43c4-9a59-273555f2ee28 service neutron] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:17:04.678489 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 927/1849] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:17:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 239 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:17:04.680285 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8354bb-3c63-43c4-9a59-273555f2ee28 service neutron] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:17:04.709471 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-5a4ad291-b22a-4bce-978d-b601f6318728 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a14634b2ca3446d3be99cbf082ad0cbc'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:17:04.720802 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8354bb-3c63-43c4-9a59-273555f2ee28 service neutron] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:17:04.730793 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-5a4ad291-b22a-4bce-978d-b601f6318728 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a14634b2ca3446d3be99cbf082ad0cbc'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:17:04.735172 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382f4c5d-5dc4-4a61-a075-7f0281079809 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:17:04.735968 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382f4c5d-5dc4-4a61-a075-7f0281079809 admin admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:04.736250 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382f4c5d-5dc4-4a61-a075-7f0281079809 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:04.736541 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382f4c5d-5dc4-4a61-a075-7f0281079809 admin admin] PATH_INFO: `/v3/domains` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:04.737542 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382f4c5d-5dc4-4a61-a075-7f0281079809 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:17:04.738667 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382f4c5d-5dc4-4a61-a075-7f0281079809 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:17:04.742162 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382f4c5d-5dc4-4a61-a075-7f0281079809 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:17:04.750987 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 923/1850] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Aug 24 06:17:04 2022] DELETE /identity/v3/users/a14634b2ca3446d3be99cbf082ad0cbc => generated 0 bytes in 508 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:17:04.755445 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b631cf4-9719-4fa5-a77f-eb0dcb2146b0 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:04.755730 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b631cf4-9719-4fa5-a77f-eb0dcb2146b0 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:04.756002 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b631cf4-9719-4fa5-a77f-eb0dcb2146b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:04.762695 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 928/1851] 10.222.0.62 () {58 vars in 1153 bytes} [Wed Aug 24 06:17:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:17:04.787266 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382f4c5d-5dc4-4a61-a075-7f0281079809 admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:17:04.791978 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-382f4c5d-5dc4-4a61-a075-7f0281079809 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:17:04.792110 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-8b631cf4-9719-4fa5-a77f-eb0dcb2146b0 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:17:04.792110 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-8b631cf4-9719-4fa5-a77f-eb0dcb2146b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:17:04.849023 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382f4c5d-5dc4-4a61-a075-7f0281079809 admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:17:04.889909 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 924/1852] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:17:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 06:17:04.937862 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b631cf4-9719-4fa5-a77f-eb0dcb2146b0 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:17:04.945994 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-8b631cf4-9719-4fa5-a77f-eb0dcb2146b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:17:04.983173 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384435af-feff-4196-8f5a-ece0f152c5d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:17:04.983998 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384435af-feff-4196-8f5a-ece0f152c5d3 admin admin] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:04.984404 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384435af-feff-4196-8f5a-ece0f152c5d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:04.984831 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384435af-feff-4196-8f5a-ece0f152c5d3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:04.985917 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384435af-feff-4196-8f5a-ece0f152c5d3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:17:04.989752 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384435af-feff-4196-8f5a-ece0f152c5d3 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:17:05.112982 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b631cf4-9719-4fa5-a77f-eb0dcb2146b0 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:17:05.118001 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623f6774-d9e5-417e-8180-13fde6ffc219 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bb6c79db75e4d169331c02133d1ec32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:17:05.118673 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623f6774-d9e5-417e-8180-13fde6ffc219 service neutron] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:05.118977 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623f6774-d9e5-417e-8180-13fde6ffc219 service neutron] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:05.119301 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623f6774-d9e5-417e-8180-13fde6ffc219 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:05.131434 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623f6774-d9e5-417e-8180-13fde6ffc219 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:17:05.132945 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623f6774-d9e5-417e-8180-13fde6ffc219 service neutron] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:17:05.151644 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 929/1853] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Aug 24 06:17:04 2022] POST /identity/v3/projects => generated 387 bytes in 365 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:17:05.152651 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 925/1854] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:17:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:17:05.155878 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384435af-feff-4196-8f5a-ece0f152c5d3 admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:17:05.269158 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384435af-feff-4196-8f5a-ece0f152c5d3 admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:17:05.294174 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1e2d42-25bd-4569-a628-a0301ea5b6af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:17:05.297835 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1e2d42-25bd-4569-a628-a0301ea5b6af admin admin] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:05.299057 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1e2d42-25bd-4569-a628-a0301ea5b6af admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:05.302096 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1e2d42-25bd-4569-a628-a0301ea5b6af admin admin] PATH_INFO: `/v3/users` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:05.302775 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1e2d42-25bd-4569-a628-a0301ea5b6af admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:17:05.311096 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1e2d42-25bd-4569-a628-a0301ea5b6af admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:17:05.321486 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:17:05.362245 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:17:05.462353 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 930/1855] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Aug 24 06:17:05 2022] POST /identity/v3/users => generated 294 bytes in 307 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 06:17:05.468152 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623f6774-d9e5-417e-8180-13fde6ffc219 service neutron] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:17:05.496105 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623f6774-d9e5-417e-8180-13fde6ffc219 service neutron] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:17:05.510315 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a627689e-77d9-43a0-9d36-b0e9b6104683 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:17:05.510315 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a627689e-77d9-43a0-9d36-b0e9b6104683 admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:05.510647 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a627689e-77d9-43a0-9d36-b0e9b6104683 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:05.510647 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a627689e-77d9-43a0-9d36-b0e9b6104683 admin admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:05.511380 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a627689e-77d9-43a0-9d36-b0e9b6104683 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:17:05.513038 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a627689e-77d9-43a0-9d36-b0e9b6104683 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:17:05.539032 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 926/1856] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Aug 24 06:17:05 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:17:05.542709 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1e2d42-25bd-4569-a628-a0301ea5b6af admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:17:05.561235 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1e2d42-25bd-4569-a628-a0301ea5b6af admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:17:05.594257 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d821dd-359b-4bd2-b34d-eebed1381210 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:17:05.604249 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d821dd-359b-4bd2-b34d-eebed1381210 admin admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:05.604249 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d821dd-359b-4bd2-b34d-eebed1381210 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:05.604249 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d821dd-359b-4bd2-b34d-eebed1381210 admin admin] PATH_INFO: `/v3/roles` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:05.604249 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d821dd-359b-4bd2-b34d-eebed1381210 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:17:05.604249 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d821dd-359b-4bd2-b34d-eebed1381210 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:17:05.637882 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 931/1857] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Aug 24 06:17:05 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:17:05.648558 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a627689e-77d9-43a0-9d36-b0e9b6104683 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:17:05.678159 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a627689e-77d9-43a0-9d36-b0e9b6104683 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:17:05.691184 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa48533-873e-4a69-aed0-2f768667ef16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:17:05.692185 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa48533-873e-4a69-aed0-2f768667ef16 admin admin] REQUEST_METHOD: `PUT` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:05.692681 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa48533-873e-4a69-aed0-2f768667ef16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:05.693464 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa48533-873e-4a69-aed0-2f768667ef16 admin admin] PATH_INFO: `/v3/projects/f31624517bb9405db0ccb057a8f7b882/users/2c4f515e937f4d34b41acf1a697d8540/roles/580aa7dbb38d4cff8f9d1bb8983f99ef` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:05.735424 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa48533-873e-4a69-aed0-2f768667ef16 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f31624517bb9405db0ccb057a8f7b882, user_id=2c4f515e937f4d34b41acf1a697d8540, role_id=580aa7dbb38d4cff8f9d1bb8983f99ef)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:17:05.741912 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa48533-873e-4a69-aed0-2f768667ef16 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:17:05.835696 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 927/1858] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Aug 24 06:17:05 2022] PUT /identity/v3/projects/f31624517bb9405db0ccb057a8f7b882/users/2c4f515e937f4d34b41acf1a697d8540/roles/580aa7dbb38d4cff8f9d1bb8983f99ef => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 06:17:05.847664 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5982550b-1fe8-4784-9173-d96983d2e3a9 None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:05.848140 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5982550b-1fe8-4784-9173-d96983d2e3a9 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:05.849152 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5982550b-1fe8-4784-9173-d96983d2e3a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:05.923342 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-5982550b-1fe8-4784-9173-d96983d2e3a9 None None] MFA Rules not processed for user `2c4f515e937f4d34b41acf1a697d8540`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:17:05.925518 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-5982550b-1fe8-4784-9173-d96983d2e3a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:17:06.062442 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa48533-873e-4a69-aed0-2f768667ef16 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:17:06.066418 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-dfa48533-873e-4a69-aed0-2f768667ef16 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:17:06.243072 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 932/1859] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:17:05 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 397 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:17:06.249563 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5982550b-1fe8-4784-9173-d96983d2e3a9 None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:17:06.330057 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa48533-873e-4a69-aed0-2f768667ef16 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:17:06.351305 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3494acc2-beea-4d4a-b841-af47a7c99fa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:17:06.354374 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3494acc2-beea-4d4a-b841-af47a7c99fa1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:06.355833 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3494acc2-beea-4d4a-b841-af47a7c99fa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:06.356149 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3494acc2-beea-4d4a-b841-af47a7c99fa1 admin admin] PATH_INFO: `/v3/projects/06831a6993814effa495a95dd0bdca0e` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:06.360257 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3494acc2-beea-4d4a-b841-af47a7c99fa1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=06831a6993814effa495a95dd0bdca0e)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:17:06.369711 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3494acc2-beea-4d4a-b841-af47a7c99fa1 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:17:06.417162 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:17:06.424285 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5982550b-1fe8-4784-9173-d96983d2e3a9 None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:17:06.430388 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7feb31e5-cdc4-4570-946c-cec6dddd7527 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:17:06.431127 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7feb31e5-cdc4-4570-946c-cec6dddd7527 admin admin] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:06.431359 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7feb31e5-cdc4-4570-946c-cec6dddd7527 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:06.431651 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7feb31e5-cdc4-4570-946c-cec6dddd7527 admin admin] PATH_INFO: `/v3/projects` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:06.433702 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7feb31e5-cdc4-4570-946c-cec6dddd7527 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:17:06.442388 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7feb31e5-cdc4-4570-946c-cec6dddd7527 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:17:06.539402 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-3494acc2-beea-4d4a-b841-af47a7c99fa1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '06831a6993814effa495a95dd0bdca0e', 'request_id': 'req-3494acc2-beea-4d4a-b841-af47a7c99fa1'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:17:06.546188 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-3494acc2-beea-4d4a-b841-af47a7c99fa1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '06831a6993814effa495a95dd0bdca0e', 'request_id': 'req-3494acc2-beea-4d4a-b841-af47a7c99fa1'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:17:06.554469 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-3494acc2-beea-4d4a-b841-af47a7c99fa1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '06831a6993814effa495a95dd0bdca0e', 'request_id': 'req-3494acc2-beea-4d4a-b841-af47a7c99fa1'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:17:06.569243 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 933/1860] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Aug 24 06:17:06 2022] POST /identity/v3/projects => generated 387 bytes in 322 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 06:17:06.570416 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7feb31e5-cdc4-4570-946c-cec6dddd7527 admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:17:06.572337 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-7feb31e5-cdc4-4570-946c-cec6dddd7527 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:17:06.647168 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-3494acc2-beea-4d4a-b841-af47a7c99fa1 admin admin] The token cache is being invalidate because project 06831a6993814effa495a95dd0bdca0e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81852) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:17:06.647716 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-3494acc2-beea-4d4a-b841-af47a7c99fa1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:17:06.648942 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-3494acc2-beea-4d4a-b841-af47a7c99fa1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:17:06.651162 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 928/1861] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Aug 24 06:17:06 2022] DELETE /identity/v3/projects/06831a6993814effa495a95dd0bdca0e => generated 0 bytes in 589 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:17:06.652197 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7feb31e5-cdc4-4570-946c-cec6dddd7527 admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:17:06.652370 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3494acc2-beea-4d4a-b841-af47a7c99fa1 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:17:06.654037 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-3494acc2-beea-4d4a-b841-af47a7c99fa1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:17:06.660801 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a464223-52d3-4162-bd31-67fb62602f9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:17:06.661642 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a464223-52d3-4162-bd31-67fb62602f9b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:06.661905 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a464223-52d3-4162-bd31-67fb62602f9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:06.662940 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a464223-52d3-4162-bd31-67fb62602f9b admin admin] PATH_INFO: `/v3/projects/fa97a9b99b8e448eb6e795b3f02630ab` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:06.664236 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a464223-52d3-4162-bd31-67fb62602f9b admin admin] RBAC: Authorizing `identity:delete_project(project_id=fa97a9b99b8e448eb6e795b3f02630ab)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:17:06.666328 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a464223-52d3-4162-bd31-67fb62602f9b admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:17:06.744408 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3494acc2-beea-4d4a-b841-af47a7c99fa1 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:17:06.749913 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45d83b6-19c5-4e4e-ad9c-8a1192132276 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:17:06.750607 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45d83b6-19c5-4e4e-ad9c-8a1192132276 admin admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:06.750921 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45d83b6-19c5-4e4e-ad9c-8a1192132276 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:06.751203 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45d83b6-19c5-4e4e-ad9c-8a1192132276 admin admin] PATH_INFO: `/v3/users` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:06.751915 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45d83b6-19c5-4e4e-ad9c-8a1192132276 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:17:06.754569 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45d83b6-19c5-4e4e-ad9c-8a1192132276 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:17:06.783025 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-1a464223-52d3-4162-bd31-67fb62602f9b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fa97a9b99b8e448eb6e795b3f02630ab', 'request_id': 'req-1a464223-52d3-4162-bd31-67fb62602f9b'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:17:06.791732 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-1a464223-52d3-4162-bd31-67fb62602f9b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fa97a9b99b8e448eb6e795b3f02630ab', 'request_id': 'req-1a464223-52d3-4162-bd31-67fb62602f9b'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:17:06.801749 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-1a464223-52d3-4162-bd31-67fb62602f9b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fa97a9b99b8e448eb6e795b3f02630ab', 'request_id': 'req-1a464223-52d3-4162-bd31-67fb62602f9b'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:17:06.847028 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 929/1862] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Aug 24 06:17:06 2022] POST /identity/v3/users => generated 300 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:17:06.849548 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45d83b6-19c5-4e4e-ad9c-8a1192132276 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:17:06.851784 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-d45d83b6-19c5-4e4e-ad9c-8a1192132276 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:17:06.867276 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45d83b6-19c5-4e4e-ad9c-8a1192132276 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:17:06.876736 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-1a464223-52d3-4162-bd31-67fb62602f9b admin admin] The token cache is being invalidate because project fa97a9b99b8e448eb6e795b3f02630ab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81851) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:17:06.876938 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-1a464223-52d3-4162-bd31-67fb62602f9b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:17:06.877814 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-1a464223-52d3-4162-bd31-67fb62602f9b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:17:06.880016 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 934/1863] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Aug 24 06:17:06 2022] DELETE /identity/v3/projects/fa97a9b99b8e448eb6e795b3f02630ab => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:17:06.881011 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a464223-52d3-4162-bd31-67fb62602f9b admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:17:06.882357 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-1a464223-52d3-4162-bd31-67fb62602f9b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:17:06.950810 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343bf29c-4012-4b66-9969-b525bc9e8cd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:17:06.951961 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343bf29c-4012-4b66-9969-b525bc9e8cd7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:06.952629 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343bf29c-4012-4b66-9969-b525bc9e8cd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:06.952939 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343bf29c-4012-4b66-9969-b525bc9e8cd7 admin admin] PATH_INFO: `/v3/users/4df37ae51a5e4d9ab3309e053dbccdab` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:06.955402 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343bf29c-4012-4b66-9969-b525bc9e8cd7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4df37ae51a5e4d9ab3309e053dbccdab)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:17:06.957500 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343bf29c-4012-4b66-9969-b525bc9e8cd7 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:17:06.965750 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a464223-52d3-4162-bd31-67fb62602f9b admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:17:06.970874 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6698025f-9c5e-462e-85f9-9c8d5dbace0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:17:06.971509 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6698025f-9c5e-462e-85f9-9c8d5dbace0e admin admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:06.975629 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6698025f-9c5e-462e-85f9-9c8d5dbace0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:06.976212 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6698025f-9c5e-462e-85f9-9c8d5dbace0e admin admin] PATH_INFO: `/v3/roles` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:06.979892 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:17:06.979892 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6698025f-9c5e-462e-85f9-9c8d5dbace0e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:17:06.982458 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6698025f-9c5e-462e-85f9-9c8d5dbace0e admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:17:07.001515 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 935/1864] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Aug 24 06:17:06 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:17:07.006579 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79adbd0d-cf6f-45b0-9459-2a2be1685885 None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:07.006904 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79adbd0d-cf6f-45b0-9459-2a2be1685885 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:07.007237 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79adbd0d-cf6f-45b0-9459-2a2be1685885 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:07.032627 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-79adbd0d-cf6f-45b0-9459-2a2be1685885 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:17:07.033848 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-79adbd0d-cf6f-45b0-9459-2a2be1685885 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:17:07.047740 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 936/1865] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:17:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:17:07.048380 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79adbd0d-cf6f-45b0-9459-2a2be1685885 None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:17:07.061134 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79adbd0d-cf6f-45b0-9459-2a2be1685885 None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:17:07.068415 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf29fd5-d986-43c9-beb0-4d1f5363b421 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:17:07.069189 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf29fd5-d986-43c9-beb0-4d1f5363b421 admin admin] REQUEST_METHOD: `PUT` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:07.069695 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf29fd5-d986-43c9-beb0-4d1f5363b421 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:07.070031 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf29fd5-d986-43c9-beb0-4d1f5363b421 admin admin] PATH_INFO: `/v3/projects/b16ede2290884266a93d6bf904c5d21f/users/cf968f742e4e40188e0a91e0b2b93eb1/roles/49e47cd157d743f6905f1db07ec878a7` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:07.087051 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf29fd5-d986-43c9-beb0-4d1f5363b421 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b16ede2290884266a93d6bf904c5d21f, user_id=cf968f742e4e40188e0a91e0b2b93eb1, role_id=49e47cd157d743f6905f1db07ec878a7)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:17:07.089772 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf29fd5-d986-43c9-beb0-4d1f5363b421 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:17:07.199355 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 937/1866] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Aug 24 06:17:07 2022] PUT /identity/v3/projects/b16ede2290884266a93d6bf904c5d21f/users/cf968f742e4e40188e0a91e0b2b93eb1/roles/49e47cd157d743f6905f1db07ec878a7 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:17:07.200585 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323babaa-31f3-468e-9524-402879327373 None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:07.200954 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323babaa-31f3-468e-9524-402879327373 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:07.201139 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323babaa-31f3-468e-9524-402879327373 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:07.209321 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-343bf29c-4012-4b66-9969-b525bc9e8cd7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4df37ae51a5e4d9ab3309e053dbccdab'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:17:07.223339 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-343bf29c-4012-4b66-9969-b525bc9e8cd7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4df37ae51a5e4d9ab3309e053dbccdab'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:17:07.224594 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-343bf29c-4012-4b66-9969-b525bc9e8cd7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4df37ae51a5e4d9ab3309e053dbccdab'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:17:07.251849 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-323babaa-31f3-468e-9524-402879327373 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:17:07.255227 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-323babaa-31f3-468e-9524-402879327373 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:17:07.282010 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-343bf29c-4012-4b66-9969-b525bc9e8cd7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4df37ae51a5e4d9ab3309e053dbccdab'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:17:07.315282 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-343bf29c-4012-4b66-9969-b525bc9e8cd7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4df37ae51a5e4d9ab3309e053dbccdab'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:17:07.334781 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 930/1867] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Aug 24 06:17:06 2022] DELETE /identity/v3/users/4df37ae51a5e4d9ab3309e053dbccdab => generated 0 bytes in 487 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:17:07.336638 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343bf29c-4012-4b66-9969-b525bc9e8cd7 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:17:07.345671 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-343bf29c-4012-4b66-9969-b525bc9e8cd7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:17:07.387650 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 938/1868] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:17:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:17:07.393012 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323babaa-31f3-468e-9524-402879327373 None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:17:07.413761 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323babaa-31f3-468e-9524-402879327373 None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:17:07.510388 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffccbabe-7888-4f66-a2a2-e9332e08a997 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:17:07.511084 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffccbabe-7888-4f66-a2a2-e9332e08a997 admin admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:07.511362 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffccbabe-7888-4f66-a2a2-e9332e08a997 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:07.511928 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffccbabe-7888-4f66-a2a2-e9332e08a997 admin admin] PATH_INFO: `/v3/domains` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:07.512485 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffccbabe-7888-4f66-a2a2-e9332e08a997 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:17:07.512899 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffccbabe-7888-4f66-a2a2-e9332e08a997 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:17:07.528719 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffccbabe-7888-4f66-a2a2-e9332e08a997 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:17:07.537741 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343bf29c-4012-4b66-9969-b525bc9e8cd7 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:17:07.544423 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbbb4c2-0d80-4da6-addd-7ceb951218fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:17:07.545764 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbbb4c2-0d80-4da6-addd-7ceb951218fa admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:07.546430 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbbb4c2-0d80-4da6-addd-7ceb951218fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:07.547002 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbbb4c2-0d80-4da6-addd-7ceb951218fa admin admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:07.547807 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbbb4c2-0d80-4da6-addd-7ceb951218fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:17:07.551785 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbbb4c2-0d80-4da6-addd-7ceb951218fa admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:17:07.572990 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 939/1869] 10.222.0.62 () {58 vars in 1153 bytes} [Wed Aug 24 06:17:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:17:07.575213 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 931/1870] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Aug 24 06:17:07 2022] GET /identity/v3/roles => generated 1420 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:17:07.577479 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffccbabe-7888-4f66-a2a2-e9332e08a997 admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:17:07.579370 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbbb4c2-0d80-4da6-addd-7ceb951218fa admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:17:07.590454 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffccbabe-7888-4f66-a2a2-e9332e08a997 admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:17:07.599629 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e9a556-91f1-4e15-99cd-4152e3401228 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:17:07.600558 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e9a556-91f1-4e15-99cd-4152e3401228 admin admin] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:07.600992 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e9a556-91f1-4e15-99cd-4152e3401228 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:07.601273 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e9a556-91f1-4e15-99cd-4152e3401228 admin admin] PATH_INFO: `/v3/projects` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:07.601727 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e9a556-91f1-4e15-99cd-4152e3401228 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:17:07.604808 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e9a556-91f1-4e15-99cd-4152e3401228 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:17:07.617808 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbbb4c2-0d80-4da6-addd-7ceb951218fa admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:17:07.647790 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba76b50f-38a7-4cd7-bbe7-b620035f85ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:17:07.648476 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba76b50f-38a7-4cd7-bbe7-b620035f85ad admin admin] REQUEST_METHOD: `PUT` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:07.648785 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba76b50f-38a7-4cd7-bbe7-b620035f85ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:07.649096 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba76b50f-38a7-4cd7-bbe7-b620035f85ad admin admin] PATH_INFO: `/v3/projects/b16ede2290884266a93d6bf904c5d21f/users/cf968f742e4e40188e0a91e0b2b93eb1/roles/49e47cd157d743f6905f1db07ec878a7` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:07.657710 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba76b50f-38a7-4cd7-bbe7-b620035f85ad admin admin] RBAC: Authorizing `identity:create_grant(project_id=b16ede2290884266a93d6bf904c5d21f, user_id=cf968f742e4e40188e0a91e0b2b93eb1, role_id=49e47cd157d743f6905f1db07ec878a7)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:17:07.662119 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba76b50f-38a7-4cd7-bbe7-b620035f85ad admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:17:07.676298 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:17:07.725692 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 932/1871] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Aug 24 06:17:07 2022] PUT /identity/v3/projects/b16ede2290884266a93d6bf904c5d21f/users/cf968f742e4e40188e0a91e0b2b93eb1/roles/49e47cd157d743f6905f1db07ec878a7 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 06:17:07.727243 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba76b50f-38a7-4cd7-bbe7-b620035f85ad admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:17:07.732865 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-ba76b50f-38a7-4cd7-bbe7-b620035f85ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:17:07.793972 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 940/1872] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Aug 24 06:17:07 2022] POST /identity/v3/projects => generated 383 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 06:17:07.796095 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228b5f41-5aa7-44bf-92ab-a493f4ba06b6 None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:07.796766 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228b5f41-5aa7-44bf-92ab-a493f4ba06b6 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:07.797178 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228b5f41-5aa7-44bf-92ab-a493f4ba06b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:07.845840 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-228b5f41-5aa7-44bf-92ab-a493f4ba06b6 None None] MFA Rules not processed for user `cf968f742e4e40188e0a91e0b2b93eb1`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:17:07.848164 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-228b5f41-5aa7-44bf-92ab-a493f4ba06b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:17:07.892706 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba76b50f-38a7-4cd7-bbe7-b620035f85ad admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:17:07.949362 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 941/1873] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:17:07 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:17:07.950897 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228b5f41-5aa7-44bf-92ab-a493f4ba06b6 None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:17:07.971640 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871139cf-d05e-42ec-86db-b00c0f48b912 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:17:07.973307 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871139cf-d05e-42ec-86db-b00c0f48b912 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:07.973875 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871139cf-d05e-42ec-86db-b00c0f48b912 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:07.974322 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871139cf-d05e-42ec-86db-b00c0f48b912 admin admin] PATH_INFO: `/v3/users/15bda6a0e79c4a7e89f9b9527b9cd60a` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:07.975675 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871139cf-d05e-42ec-86db-b00c0f48b912 admin admin] RBAC: Authorizing `identity:delete_user(user_id=15bda6a0e79c4a7e89f9b9527b9cd60a)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:17:07.979961 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871139cf-d05e-42ec-86db-b00c0f48b912 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:17:08.040141 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:17:08.045836 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228b5f41-5aa7-44bf-92ab-a493f4ba06b6 None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:17:08.056877 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653b8f99-fc51-418a-b6f2-69307e7ee620 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:17:08.060238 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653b8f99-fc51-418a-b6f2-69307e7ee620 admin admin] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:08.061009 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653b8f99-fc51-418a-b6f2-69307e7ee620 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:08.061541 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653b8f99-fc51-418a-b6f2-69307e7ee620 admin admin] PATH_INFO: `/v3/users` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:08.062544 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653b8f99-fc51-418a-b6f2-69307e7ee620 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:17:08.065654 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653b8f99-fc51-418a-b6f2-69307e7ee620 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:17:08.157117 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 942/1874] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Aug 24 06:17:07 2022] POST /identity/v3/users => generated 292 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:17:08.158286 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653b8f99-fc51-418a-b6f2-69307e7ee620 admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:17:08.162336 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-653b8f99-fc51-418a-b6f2-69307e7ee620 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:17:08.230624 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653b8f99-fc51-418a-b6f2-69307e7ee620 admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:17:08.239801 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-871139cf-d05e-42ec-86db-b00c0f48b912 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '15bda6a0e79c4a7e89f9b9527b9cd60a'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:17:08.250445 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf67a5bf-45a3-4e81-9d4a-e980f8fca0e7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c4def55ac0e48b29836bc5c64bc7402', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:17:08.272214 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf67a5bf-45a3-4e81-9d4a-e980f8fca0e7 service cinder] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:08.272214 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf67a5bf-45a3-4e81-9d4a-e980f8fca0e7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:08.272214 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf67a5bf-45a3-4e81-9d4a-e980f8fca0e7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:08.273160 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-871139cf-d05e-42ec-86db-b00c0f48b912 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '15bda6a0e79c4a7e89f9b9527b9cd60a'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:17:08.273160 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-871139cf-d05e-42ec-86db-b00c0f48b912 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '15bda6a0e79c4a7e89f9b9527b9cd60a'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:17:08.273468 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf67a5bf-45a3-4e81-9d4a-e980f8fca0e7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:17:08.273468 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf67a5bf-45a3-4e81-9d4a-e980f8fca0e7 service cinder] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:17:08.290457 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 943/1875] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:17:08 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:17:08.291728 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf67a5bf-45a3-4e81-9d4a-e980f8fca0e7 service cinder] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:17:08.295826 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-cf67a5bf-45a3-4e81-9d4a-e980f8fca0e7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:17:08.314025 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf67a5bf-45a3-4e81-9d4a-e980f8fca0e7 service cinder] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:17:08.321316 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44d1953-7810-4713-a46d-773706ffd0bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:17:08.323759 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44d1953-7810-4713-a46d-773706ffd0bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:08.323759 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44d1953-7810-4713-a46d-773706ffd0bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:08.323759 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44d1953-7810-4713-a46d-773706ffd0bd admin admin] PATH_INFO: `/v3/projects/70e9b8312ab6415296e995d7510dc832` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:08.324618 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44d1953-7810-4713-a46d-773706ffd0bd admin admin] RBAC: Authorizing `identity:delete_project(project_id=70e9b8312ab6415296e995d7510dc832)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:17:08.326002 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44d1953-7810-4713-a46d-773706ffd0bd admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:17:08.738263 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-871139cf-d05e-42ec-86db-b00c0f48b912 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '15bda6a0e79c4a7e89f9b9527b9cd60a'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:17:08.750559 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-871139cf-d05e-42ec-86db-b00c0f48b912 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '15bda6a0e79c4a7e89f9b9527b9cd60a'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:17:08.757324 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 933/1876] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Aug 24 06:17:07 2022] DELETE /identity/v3/users/15bda6a0e79c4a7e89f9b9527b9cd60a => generated 0 bytes in 1031 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:17:08.758159 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871139cf-d05e-42ec-86db-b00c0f48b912 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:17:08.759501 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-871139cf-d05e-42ec-86db-b00c0f48b912 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:17:08.767506 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:17:08.785444 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-d44d1953-7810-4713-a46d-773706ffd0bd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '70e9b8312ab6415296e995d7510dc832', 'request_id': 'req-d44d1953-7810-4713-a46d-773706ffd0bd'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:17:08.791495 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-d44d1953-7810-4713-a46d-773706ffd0bd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '70e9b8312ab6415296e995d7510dc832', 'request_id': 'req-d44d1953-7810-4713-a46d-773706ffd0bd'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:17:08.798047 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-d44d1953-7810-4713-a46d-773706ffd0bd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '70e9b8312ab6415296e995d7510dc832', 'request_id': 'req-d44d1953-7810-4713-a46d-773706ffd0bd'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:17:08.831295 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871139cf-d05e-42ec-86db-b00c0f48b912 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:17:08.893215 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed532663-59ab-411e-936e-3e174f2eebd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:17:08.894075 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-d44d1953-7810-4713-a46d-773706ffd0bd admin admin] The token cache is being invalidate because project 70e9b8312ab6415296e995d7510dc832 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81851) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:17:08.894204 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed532663-59ab-411e-936e-3e174f2eebd3 admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:08.894494 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed532663-59ab-411e-936e-3e174f2eebd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:08.894637 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-d44d1953-7810-4713-a46d-773706ffd0bd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:17:08.894902 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed532663-59ab-411e-936e-3e174f2eebd3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:08.895586 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-d44d1953-7810-4713-a46d-773706ffd0bd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:17:08.895769 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed532663-59ab-411e-936e-3e174f2eebd3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:17:08.901451 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 944/1877] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Aug 24 06:17:08 2022] DELETE /identity/v3/projects/70e9b8312ab6415296e995d7510dc832 => generated 0 bytes in 607 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:17:08.901451 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44d1953-7810-4713-a46d-773706ffd0bd admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:17:08.901451 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-d44d1953-7810-4713-a46d-773706ffd0bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:17:08.904790 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed532663-59ab-411e-936e-3e174f2eebd3 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:17:08.914615 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44d1953-7810-4713-a46d-773706ffd0bd admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:17:08.919654 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c4ff49-2231-4b5a-9b5a-7cf42b271bb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:17:08.920289 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c4ff49-2231-4b5a-9b5a-7cf42b271bb7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:08.920903 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c4ff49-2231-4b5a-9b5a-7cf42b271bb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:08.921154 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c4ff49-2231-4b5a-9b5a-7cf42b271bb7 admin admin] PATH_INFO: `/v3/users/2c4f515e937f4d34b41acf1a697d8540` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:08.922440 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c4ff49-2231-4b5a-9b5a-7cf42b271bb7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2c4f515e937f4d34b41acf1a697d8540)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:17:08.925575 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c4ff49-2231-4b5a-9b5a-7cf42b271bb7 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:17:08.927416 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 934/1878] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Aug 24 06:17:08 2022] GET /identity/v3/roles => generated 1420 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:17:08.929007 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a14cf2-82b3-498b-a967-913953363e56 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:08.929474 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a14cf2-82b3-498b-a967-913953363e56 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:08.929787 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a14cf2-82b3-498b-a967-913953363e56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:08.955940 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-17a14cf2-82b3-498b-a967-913953363e56 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:17:08.957503 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-17a14cf2-82b3-498b-a967-913953363e56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:17:08.965855 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 935/1879] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:17:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:17:08.967170 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b0fe9d-75a5-42d4-bca1-9ccc55381848 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:08.967454 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b0fe9d-75a5-42d4-bca1-9ccc55381848 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:08.967722 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b0fe9d-75a5-42d4-bca1-9ccc55381848 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:08.990852 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-f2b0fe9d-75a5-42d4-bca1-9ccc55381848 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:17:08.991975 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-f2b0fe9d-75a5-42d4-bca1-9ccc55381848 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:17:09.000254 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 936/1880] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:17:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:17:09.001652 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b0fe9d-75a5-42d4-bca1-9ccc55381848 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:17:09.003832 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-f2b0fe9d-75a5-42d4-bca1-9ccc55381848 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:17:09.016472 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b0fe9d-75a5-42d4-bca1-9ccc55381848 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:17:09.021851 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa14a268-0ac6-441c-96c0-6ee0033a5be8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:17:09.022762 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa14a268-0ac6-441c-96c0-6ee0033a5be8 admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:09.023125 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa14a268-0ac6-441c-96c0-6ee0033a5be8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:09.023467 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa14a268-0ac6-441c-96c0-6ee0033a5be8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:09.024124 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa14a268-0ac6-441c-96c0-6ee0033a5be8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:17:09.025802 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa14a268-0ac6-441c-96c0-6ee0033a5be8 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:17:09.039063 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 937/1881] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Aug 24 06:17:09 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:17:09.039063 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa14a268-0ac6-441c-96c0-6ee0033a5be8 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:17:09.039063 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-fa14a268-0ac6-441c-96c0-6ee0033a5be8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:17:09.097456 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa14a268-0ac6-441c-96c0-6ee0033a5be8 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:17:09.106930 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fe9769-3b92-4a23-9a7c-e2ae6ace6751 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bb6c79db75e4d169331c02133d1ec32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:17:09.106930 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fe9769-3b92-4a23-9a7c-e2ae6ace6751 service neutron] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:09.106930 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fe9769-3b92-4a23-9a7c-e2ae6ace6751 service neutron] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:09.106930 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fe9769-3b92-4a23-9a7c-e2ae6ace6751 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:09.111250 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40fe9769-3b92-4a23-9a7c-e2ae6ace6751 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:17:09.113182 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40fe9769-3b92-4a23-9a7c-e2ae6ace6751 service neutron] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:17:09.122295 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 938/1882] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:17:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:17:09.123427 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9422b8d-3a3b-4726-b88e-b230217d0a4e None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:09.123701 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9422b8d-3a3b-4726-b88e-b230217d0a4e None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:09.123968 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9422b8d-3a3b-4726-b88e-b230217d0a4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:09.142432 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-59c4ff49-2231-4b5a-9b5a-7cf42b271bb7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2c4f515e937f4d34b41acf1a697d8540'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:17:09.146238 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-c9422b8d-3a3b-4726-b88e-b230217d0a4e None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:17:09.147478 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-c9422b8d-3a3b-4726-b88e-b230217d0a4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:17:09.149717 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-59c4ff49-2231-4b5a-9b5a-7cf42b271bb7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2c4f515e937f4d34b41acf1a697d8540'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:17:09.150443 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-59c4ff49-2231-4b5a-9b5a-7cf42b271bb7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2c4f515e937f4d34b41acf1a697d8540'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:17:09.151050 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-59c4ff49-2231-4b5a-9b5a-7cf42b271bb7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2c4f515e937f4d34b41acf1a697d8540'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:17:09.156977 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 939/1883] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:17:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:17:09.157568 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9422b8d-3a3b-4726-b88e-b230217d0a4e None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:17:09.158834 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-c9422b8d-3a3b-4726-b88e-b230217d0a4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:17:09.170880 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9422b8d-3a3b-4726-b88e-b230217d0a4e None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:17:09.176400 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26e5d42-b31b-4a67-b69c-f8e354d54507 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:17:09.177084 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26e5d42-b31b-4a67-b69c-f8e354d54507 admin admin] REQUEST_METHOD: `PUT` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:09.177359 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26e5d42-b31b-4a67-b69c-f8e354d54507 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:09.177658 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26e5d42-b31b-4a67-b69c-f8e354d54507 admin admin] PATH_INFO: `/v3/projects/e2adcef70b1f41d19ddee6811b47be41/users/d3a03b5f82724f37a2134554d42b2925/roles/580aa7dbb38d4cff8f9d1bb8983f99ef` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:09.192551 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26e5d42-b31b-4a67-b69c-f8e354d54507 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e2adcef70b1f41d19ddee6811b47be41, user_id=d3a03b5f82724f37a2134554d42b2925, role_id=580aa7dbb38d4cff8f9d1bb8983f99ef)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:17:09.195222 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26e5d42-b31b-4a67-b69c-f8e354d54507 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:17:09.241406 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-59c4ff49-2231-4b5a-9b5a-7cf42b271bb7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2c4f515e937f4d34b41acf1a697d8540'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:17:09.259165 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 945/1884] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Aug 24 06:17:08 2022] DELETE /identity/v3/users/2c4f515e937f4d34b41acf1a697d8540 => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:17:09.259943 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c4ff49-2231-4b5a-9b5a-7cf42b271bb7 admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:17:09.320874 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c4ff49-2231-4b5a-9b5a-7cf42b271bb7 admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:17:09.325589 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670683eb-6e3e-49de-9841-e6f0924a5acc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:17:09.326193 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670683eb-6e3e-49de-9841-e6f0924a5acc admin admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:09.326471 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670683eb-6e3e-49de-9841-e6f0924a5acc admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:09.326744 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670683eb-6e3e-49de-9841-e6f0924a5acc admin admin] PATH_INFO: `/v3/domains` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:09.327388 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670683eb-6e3e-49de-9841-e6f0924a5acc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:17:09.327692 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670683eb-6e3e-49de-9841-e6f0924a5acc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:17:09.329319 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670683eb-6e3e-49de-9841-e6f0924a5acc admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:17:09.341650 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 940/1885] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Aug 24 06:17:09 2022] PUT /identity/v3/projects/e2adcef70b1f41d19ddee6811b47be41/users/d3a03b5f82724f37a2134554d42b2925/roles/580aa7dbb38d4cff8f9d1bb8983f99ef => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:17:09.346553 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 946/1886] 10.222.0.62 () {58 vars in 1153 bytes} [Wed Aug 24 06:17:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:17:09.346553 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3fe083-e4bf-4b6c-ae3f-1b2ce493850e None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:09.346553 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3fe083-e4bf-4b6c-ae3f-1b2ce493850e None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:09.347316 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7282f6-e739-44c4-9cc7-1419ea801af0 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:09.347401 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3fe083-e4bf-4b6c-ae3f-1b2ce493850e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:09.347473 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7282f6-e739-44c4-9cc7-1419ea801af0 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:09.347473 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7282f6-e739-44c4-9cc7-1419ea801af0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:09.365716 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-7b3fe083-e4bf-4b6c-ae3f-1b2ce493850e None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:17:09.366932 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-7b3fe083-e4bf-4b6c-ae3f-1b2ce493850e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:17:09.367753 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-bf7282f6-e739-44c4-9cc7-1419ea801af0 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:17:09.368984 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-bf7282f6-e739-44c4-9cc7-1419ea801af0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:17:09.442910 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 941/1887] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:17:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:17:09.444405 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7282f6-e739-44c4-9cc7-1419ea801af0 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:17:09.446014 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 947/1888] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:17:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:17:09.455391 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7282f6-e739-44c4-9cc7-1419ea801af0 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:17:09.460672 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efd533e-4116-4220-87fd-ecec221d0304 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:17:09.462739 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efd533e-4116-4220-87fd-ecec221d0304 admin admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:09.462739 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efd533e-4116-4220-87fd-ecec221d0304 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:09.463005 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3fe083-e4bf-4b6c-ae3f-1b2ce493850e None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:17:09.463097 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efd533e-4116-4220-87fd-ecec221d0304 admin admin] PATH_INFO: `/v3/projects` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:09.463097 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efd533e-4116-4220-87fd-ecec221d0304 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:17:09.464844 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-7b3fe083-e4bf-4b6c-ae3f-1b2ce493850e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:17:09.465484 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efd533e-4116-4220-87fd-ecec221d0304 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:17:09.505655 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=97,keystone:INSERT=2,keystone:DELETE=1 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:17:09.537051 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3fe083-e4bf-4b6c-ae3f-1b2ce493850e None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:17:09.540682 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28471d5-42dc-48d4-92fc-9bea1f9542be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bb6c79db75e4d169331c02133d1ec32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:17:09.541298 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28471d5-42dc-48d4-92fc-9bea1f9542be service neutron] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:09.541572 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28471d5-42dc-48d4-92fc-9bea1f9542be service neutron] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:09.541846 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28471d5-42dc-48d4-92fc-9bea1f9542be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:09.552474 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28471d5-42dc-48d4-92fc-9bea1f9542be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:17:09.554193 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28471d5-42dc-48d4-92fc-9bea1f9542be service neutron] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:17:09.563923 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 948/1889] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:17:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:17:09.564771 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28471d5-42dc-48d4-92fc-9bea1f9542be service neutron] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:17:09.566231 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-b28471d5-42dc-48d4-92fc-9bea1f9542be service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:17:09.577239 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28471d5-42dc-48d4-92fc-9bea1f9542be service neutron] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:17:09.580528 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 942/1890] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Aug 24 06:17:09 2022] POST /identity/v3/projects => generated 409 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:17:09.582982 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efd533e-4116-4220-87fd-ecec221d0304 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:17:09.661843 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f365d08a-65fd-486c-919e-8dcd4f9adabb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bb6c79db75e4d169331c02133d1ec32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:17:09.663207 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f365d08a-65fd-486c-919e-8dcd4f9adabb service neutron] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:09.663487 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f365d08a-65fd-486c-919e-8dcd4f9adabb service neutron] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:09.663760 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f365d08a-65fd-486c-919e-8dcd4f9adabb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:09.692748 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efd533e-4116-4220-87fd-ecec221d0304 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:17:09.700734 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1602723-b451-4278-a28e-df8257e04e05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:17:09.701695 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1602723-b451-4278-a28e-df8257e04e05 admin admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:09.708146 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1602723-b451-4278-a28e-df8257e04e05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:09.708146 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1602723-b451-4278-a28e-df8257e04e05 admin admin] PATH_INFO: `/v3/users` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:09.708146 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1602723-b451-4278-a28e-df8257e04e05 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:17:09.710429 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1602723-b451-4278-a28e-df8257e04e05 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:17:09.746293 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f365d08a-65fd-486c-919e-8dcd4f9adabb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:17:09.747742 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f365d08a-65fd-486c-919e-8dcd4f9adabb service neutron] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:17:09.758368 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 949/1891] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:17:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:17:09.759247 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f365d08a-65fd-486c-919e-8dcd4f9adabb service neutron] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:17:09.760606 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-f365d08a-65fd-486c-919e-8dcd4f9adabb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:17:09.772793 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:17:09.773968 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f365d08a-65fd-486c-919e-8dcd4f9adabb service neutron] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:17:09.778652 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e464f73-1f3b-4b06-9dcb-a8bb8de36ff4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:17:09.784160 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e464f73-1f3b-4b06-9dcb-a8bb8de36ff4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:09.784160 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e464f73-1f3b-4b06-9dcb-a8bb8de36ff4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:09.784160 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e464f73-1f3b-4b06-9dcb-a8bb8de36ff4 admin admin] PATH_INFO: `/v3/projects/c779507bb66d449e8550c9eb45500474` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:09.784160 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e464f73-1f3b-4b06-9dcb-a8bb8de36ff4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c779507bb66d449e8550c9eb45500474)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:17:09.784160 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e464f73-1f3b-4b06-9dcb-a8bb8de36ff4 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:17:09.812611 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 943/1892] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Aug 24 06:17:09 2022] POST /identity/v3/users => generated 305 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 06:17:09.813636 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1602723-b451-4278-a28e-df8257e04e05 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:17:09.828276 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1602723-b451-4278-a28e-df8257e04e05 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:17:09.833668 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26e2cc0-3ed5-44ff-b31b-0f10d8d6f1b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:17:09.834280 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26e2cc0-3ed5-44ff-b31b-0f10d8d6f1b6 admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:09.834560 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26e2cc0-3ed5-44ff-b31b-0f10d8d6f1b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:09.834830 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26e2cc0-3ed5-44ff-b31b-0f10d8d6f1b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:09.835469 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26e2cc0-3ed5-44ff-b31b-0f10d8d6f1b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:17:09.837096 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26e2cc0-3ed5-44ff-b31b-0f10d8d6f1b6 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:17:09.851365 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 944/1893] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Aug 24 06:17:09 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:17:09.852338 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26e2cc0-3ed5-44ff-b31b-0f10d8d6f1b6 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:17:09.875334 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26e2cc0-3ed5-44ff-b31b-0f10d8d6f1b6 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:17:09.880073 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d6c15c-35f0-42d3-9820-d20b929d5439 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:17:09.880703 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d6c15c-35f0-42d3-9820-d20b929d5439 admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:09.880980 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d6c15c-35f0-42d3-9820-d20b929d5439 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:09.881249 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d6c15c-35f0-42d3-9820-d20b929d5439 admin admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:09.881874 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d6c15c-35f0-42d3-9820-d20b929d5439 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:17:09.883459 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d6c15c-35f0-42d3-9820-d20b929d5439 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:17:09.894558 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 945/1894] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Aug 24 06:17:09 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:17:09.899065 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d6c15c-35f0-42d3-9820-d20b929d5439 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:17:09.908597 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d6c15c-35f0-42d3-9820-d20b929d5439 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:17:09.914026 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed571320-f193-4545-8b6d-a301ea044db7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:17:09.915216 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed571320-f193-4545-8b6d-a301ea044db7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:09.915216 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed571320-f193-4545-8b6d-a301ea044db7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:09.915464 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed571320-f193-4545-8b6d-a301ea044db7 admin admin] PATH_INFO: `/v3/projects/69ab491edfef411eaef0bea127a0ae73/users/0e56d56a9a80412a839f5376d7c116f7/roles/580aa7dbb38d4cff8f9d1bb8983f99ef` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:09.932483 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed571320-f193-4545-8b6d-a301ea044db7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=69ab491edfef411eaef0bea127a0ae73, user_id=0e56d56a9a80412a839f5376d7c116f7, role_id=580aa7dbb38d4cff8f9d1bb8983f99ef)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:17:09.935801 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed571320-f193-4545-8b6d-a301ea044db7 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:17:09.937664 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-1e464f73-1f3b-4b06-9dcb-a8bb8de36ff4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c779507bb66d449e8550c9eb45500474', 'request_id': 'req-1e464f73-1f3b-4b06-9dcb-a8bb8de36ff4'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:17:09.944962 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-1e464f73-1f3b-4b06-9dcb-a8bb8de36ff4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c779507bb66d449e8550c9eb45500474', 'request_id': 'req-1e464f73-1f3b-4b06-9dcb-a8bb8de36ff4'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:17:09.952755 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-1e464f73-1f3b-4b06-9dcb-a8bb8de36ff4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c779507bb66d449e8550c9eb45500474', 'request_id': 'req-1e464f73-1f3b-4b06-9dcb-a8bb8de36ff4'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:17:10.009898 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 946/1895] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Aug 24 06:17:09 2022] PUT /identity/v3/projects/69ab491edfef411eaef0bea127a0ae73/users/0e56d56a9a80412a839f5376d7c116f7/roles/580aa7dbb38d4cff8f9d1bb8983f99ef => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 06:17:10.021266 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815cc4ac-de3f-4b7e-8a55-634e50472cc0 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:10.021556 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815cc4ac-de3f-4b7e-8a55-634e50472cc0 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:10.021975 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815cc4ac-de3f-4b7e-8a55-634e50472cc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:10.037872 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-1e464f73-1f3b-4b06-9dcb-a8bb8de36ff4 admin admin] The token cache is being invalidate because project c779507bb66d449e8550c9eb45500474 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81851) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:17:10.039039 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-1e464f73-1f3b-4b06-9dcb-a8bb8de36ff4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:17:10.039632 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-1e464f73-1f3b-4b06-9dcb-a8bb8de36ff4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:17:10.044829 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 950/1896] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Aug 24 06:17:09 2022] DELETE /identity/v3/projects/c779507bb66d449e8550c9eb45500474 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:17:10.048711 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e464f73-1f3b-4b06-9dcb-a8bb8de36ff4 admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:17:10.050438 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-1e464f73-1f3b-4b06-9dcb-a8bb8de36ff4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:17:10.054617 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-815cc4ac-de3f-4b7e-8a55-634e50472cc0 None None] MFA Rules not processed for user `0e56d56a9a80412a839f5376d7c116f7`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:17:10.056952 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-815cc4ac-de3f-4b7e-8a55-634e50472cc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:17:10.126099 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e464f73-1f3b-4b06-9dcb-a8bb8de36ff4 admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:17:10.133430 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 947/1897] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:17:10 2022] POST /identity/v3/auth/tokens => generated 3286 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 06:17:10.142348 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815cc4ac-de3f-4b7e-8a55-634e50472cc0 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:17:10.145198 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-815cc4ac-de3f-4b7e-8a55-634e50472cc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:17:10.238051 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca8008b-72cf-4605-9fe4-d8076d2863bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:17:10.238700 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca8008b-72cf-4605-9fe4-d8076d2863bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:10.238980 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca8008b-72cf-4605-9fe4-d8076d2863bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:10.239258 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca8008b-72cf-4605-9fe4-d8076d2863bd admin admin] PATH_INFO: `/v3/users/54a17a48f603457995acbec718b6a3b7` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:10.240400 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca8008b-72cf-4605-9fe4-d8076d2863bd admin admin] RBAC: Authorizing `identity:delete_user(user_id=54a17a48f603457995acbec718b6a3b7)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:17:10.242408 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca8008b-72cf-4605-9fe4-d8076d2863bd admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:17:10.261000 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815cc4ac-de3f-4b7e-8a55-634e50472cc0 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:17:10.267257 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f3ad6c-2441-4e9e-b2ff-5f1f0a98373f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:17:10.267951 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f3ad6c-2441-4e9e-b2ff-5f1f0a98373f admin admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:10.268232 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f3ad6c-2441-4e9e-b2ff-5f1f0a98373f admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:10.268523 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f3ad6c-2441-4e9e-b2ff-5f1f0a98373f admin admin] PATH_INFO: `/v3/projects` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:10.269989 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f3ad6c-2441-4e9e-b2ff-5f1f0a98373f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:17:10.271915 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f3ad6c-2441-4e9e-b2ff-5f1f0a98373f admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:17:10.402463 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 948/1898] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Aug 24 06:17:10 2022] POST /identity/v3/projects => generated 405 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:17:10.403297 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f3ad6c-2441-4e9e-b2ff-5f1f0a98373f admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:17:10.404842 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-93f3ad6c-2441-4e9e-b2ff-5f1f0a98373f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:17:10.442068 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-7ca8008b-72cf-4605-9fe4-d8076d2863bd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '54a17a48f603457995acbec718b6a3b7'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:17:10.451117 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-7ca8008b-72cf-4605-9fe4-d8076d2863bd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '54a17a48f603457995acbec718b6a3b7'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:17:10.456787 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-7ca8008b-72cf-4605-9fe4-d8076d2863bd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '54a17a48f603457995acbec718b6a3b7'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:17:10.456787 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-7ca8008b-72cf-4605-9fe4-d8076d2863bd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '54a17a48f603457995acbec718b6a3b7'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:17:10.475313 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f3ad6c-2441-4e9e-b2ff-5f1f0a98373f admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:17:10.479929 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213fa75a-f0ee-4a7a-a26a-c99fc6498be8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:17:10.487278 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213fa75a-f0ee-4a7a-a26a-c99fc6498be8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:10.487278 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213fa75a-f0ee-4a7a-a26a-c99fc6498be8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:10.487278 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213fa75a-f0ee-4a7a-a26a-c99fc6498be8 admin admin] PATH_INFO: `/v3/projects/f31624517bb9405db0ccb057a8f7b882` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:10.487278 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213fa75a-f0ee-4a7a-a26a-c99fc6498be8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f31624517bb9405db0ccb057a8f7b882)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:17:10.487278 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213fa75a-f0ee-4a7a-a26a-c99fc6498be8 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:17:10.538450 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-7ca8008b-72cf-4605-9fe4-d8076d2863bd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '54a17a48f603457995acbec718b6a3b7'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:17:10.560802 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 951/1899] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Aug 24 06:17:10 2022] DELETE /identity/v3/users/54a17a48f603457995acbec718b6a3b7 => generated 0 bytes in 512 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:17:10.563671 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca8008b-72cf-4605-9fe4-d8076d2863bd admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:17:10.566458 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-7ca8008b-72cf-4605-9fe4-d8076d2863bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:17:10.626975 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-213fa75a-f0ee-4a7a-a26a-c99fc6498be8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f31624517bb9405db0ccb057a8f7b882', 'request_id': 'req-213fa75a-f0ee-4a7a-a26a-c99fc6498be8'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:17:10.638595 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-213fa75a-f0ee-4a7a-a26a-c99fc6498be8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f31624517bb9405db0ccb057a8f7b882', 'request_id': 'req-213fa75a-f0ee-4a7a-a26a-c99fc6498be8'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:17:10.638767 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca8008b-72cf-4605-9fe4-d8076d2863bd admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:17:10.641747 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-213fa75a-f0ee-4a7a-a26a-c99fc6498be8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f31624517bb9405db0ccb057a8f7b882', 'request_id': 'req-213fa75a-f0ee-4a7a-a26a-c99fc6498be8'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:17:10.698662 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8857341-5ea7-49b4-bc92-ca51e5617334 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:17:10.699281 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8857341-5ea7-49b4-bc92-ca51e5617334 admin admin] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:10.699565 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8857341-5ea7-49b4-bc92-ca51e5617334 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:10.699840 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8857341-5ea7-49b4-bc92-ca51e5617334 admin admin] PATH_INFO: `/v3/users` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:10.700478 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8857341-5ea7-49b4-bc92-ca51e5617334 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:17:10.704161 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8857341-5ea7-49b4-bc92-ca51e5617334 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:17:10.749362 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-213fa75a-f0ee-4a7a-a26a-c99fc6498be8 admin admin] The token cache is being invalidate because project f31624517bb9405db0ccb057a8f7b882 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81852) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:17:10.749836 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-213fa75a-f0ee-4a7a-a26a-c99fc6498be8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:17:10.750418 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-213fa75a-f0ee-4a7a-a26a-c99fc6498be8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:17:10.752184 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 949/1900] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Aug 24 06:17:10 2022] DELETE /identity/v3/projects/f31624517bb9405db0ccb057a8f7b882 => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:17:10.768094 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213fa75a-f0ee-4a7a-a26a-c99fc6498be8 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:17:10.781374 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-213fa75a-f0ee-4a7a-a26a-c99fc6498be8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:17:10.796275 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213fa75a-f0ee-4a7a-a26a-c99fc6498be8 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:17:10.801283 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b73387-fb03-47fa-8c9c-8bd182c9cac4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:17:10.809174 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b73387-fb03-47fa-8c9c-8bd182c9cac4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:10.809174 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b73387-fb03-47fa-8c9c-8bd182c9cac4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:10.809174 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b73387-fb03-47fa-8c9c-8bd182c9cac4 admin admin] PATH_INFO: `/v3/users/cf968f742e4e40188e0a91e0b2b93eb1` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:10.809174 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b73387-fb03-47fa-8c9c-8bd182c9cac4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cf968f742e4e40188e0a91e0b2b93eb1)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:17:10.809174 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b73387-fb03-47fa-8c9c-8bd182c9cac4 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:17:10.841707 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 952/1901] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Aug 24 06:17:10 2022] POST /identity/v3/users => generated 309 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:17:10.842827 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8857341-5ea7-49b4-bc92-ca51e5617334 admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:17:10.845178 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-c8857341-5ea7-49b4-bc92-ca51e5617334 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:17:10.857984 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8857341-5ea7-49b4-bc92-ca51e5617334 admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:17:10.862690 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff1b778-d257-4491-af93-d0966ad1e438 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:17:10.863249 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff1b778-d257-4491-af93-d0966ad1e438 admin admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:10.863554 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff1b778-d257-4491-af93-d0966ad1e438 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:10.863837 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff1b778-d257-4491-af93-d0966ad1e438 admin admin] PATH_INFO: `/v3/roles` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:10.864913 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff1b778-d257-4491-af93-d0966ad1e438 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:17:10.866171 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff1b778-d257-4491-af93-d0966ad1e438 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:17:10.893224 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 953/1902] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Aug 24 06:17:10 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:17:10.901599 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff1b778-d257-4491-af93-d0966ad1e438 admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:17:10.913941 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff1b778-d257-4491-af93-d0966ad1e438 admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:17:10.919590 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad5ca7a-2f9a-4a90-93d7-0cd386dc7e58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:17:10.920692 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad5ca7a-2f9a-4a90-93d7-0cd386dc7e58 admin admin] REQUEST_METHOD: `PUT` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:10.920976 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad5ca7a-2f9a-4a90-93d7-0cd386dc7e58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:10.921260 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad5ca7a-2f9a-4a90-93d7-0cd386dc7e58 admin admin] PATH_INFO: `/v3/projects/9f57110cd4574ca7b3eeecf20e1f0d1b/users/a9eb694acfd842369e2c398460187002/roles/49e47cd157d743f6905f1db07ec878a7` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:10.937638 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad5ca7a-2f9a-4a90-93d7-0cd386dc7e58 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f57110cd4574ca7b3eeecf20e1f0d1b, user_id=a9eb694acfd842369e2c398460187002, role_id=49e47cd157d743f6905f1db07ec878a7)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:17:10.948721 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad5ca7a-2f9a-4a90-93d7-0cd386dc7e58 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:17:10.977709 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:17:11.043068 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-20b73387-fb03-47fa-8c9c-8bd182c9cac4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cf968f742e4e40188e0a91e0b2b93eb1'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:17:11.051202 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-20b73387-fb03-47fa-8c9c-8bd182c9cac4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cf968f742e4e40188e0a91e0b2b93eb1'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:17:11.052104 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-20b73387-fb03-47fa-8c9c-8bd182c9cac4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cf968f742e4e40188e0a91e0b2b93eb1'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:17:11.061201 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 954/1903] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Aug 24 06:17:10 2022] PUT /identity/v3/projects/9f57110cd4574ca7b3eeecf20e1f0d1b/users/a9eb694acfd842369e2c398460187002/roles/49e47cd157d743f6905f1db07ec878a7 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:17:11.065259 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad5ca7a-2f9a-4a90-93d7-0cd386dc7e58 admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:17:11.132733 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad5ca7a-2f9a-4a90-93d7-0cd386dc7e58 admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:17:11.138778 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b230b9-cf08-4cd7-9b36-e42c1b5c4dac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:17:11.139420 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b230b9-cf08-4cd7-9b36-e42c1b5c4dac admin admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:11.139698 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b230b9-cf08-4cd7-9b36-e42c1b5c4dac admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:11.139970 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b230b9-cf08-4cd7-9b36-e42c1b5c4dac admin admin] PATH_INFO: `/v3/roles` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:11.141270 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b230b9-cf08-4cd7-9b36-e42c1b5c4dac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:17:11.143141 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b230b9-cf08-4cd7-9b36-e42c1b5c4dac admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:17:11.150607 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-20b73387-fb03-47fa-8c9c-8bd182c9cac4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cf968f742e4e40188e0a91e0b2b93eb1'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:17:11.154709 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 955/1904] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Aug 24 06:17:11 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 06:17:11.155781 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b230b9-cf08-4cd7-9b36-e42c1b5c4dac admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:17:11.158322 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-94b230b9-cf08-4cd7-9b36-e42c1b5c4dac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:17:11.168517 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-20b73387-fb03-47fa-8c9c-8bd182c9cac4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cf968f742e4e40188e0a91e0b2b93eb1'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:17:11.171077 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b230b9-cf08-4cd7-9b36-e42c1b5c4dac admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:17:11.174438 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 950/1905] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Aug 24 06:17:10 2022] DELETE /identity/v3/users/cf968f742e4e40188e0a91e0b2b93eb1 => generated 0 bytes in 409 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:17:11.176026 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b73387-fb03-47fa-8c9c-8bd182c9cac4 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:17:11.179674 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-20b73387-fb03-47fa-8c9c-8bd182c9cac4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:17:11.180424 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add27e72-b170-4775-b60e-2348a712bc3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:17:11.181409 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add27e72-b170-4775-b60e-2348a712bc3b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:11.181990 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add27e72-b170-4775-b60e-2348a712bc3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:11.182270 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add27e72-b170-4775-b60e-2348a712bc3b admin admin] PATH_INFO: `/v3/projects/5053dda49bb14b28b9357947315170b6` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:11.183905 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add27e72-b170-4775-b60e-2348a712bc3b admin admin] RBAC: Authorizing `identity:delete_project(project_id=5053dda49bb14b28b9357947315170b6)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:17:11.186261 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add27e72-b170-4775-b60e-2348a712bc3b admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:17:11.243278 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:17:11.255468 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b73387-fb03-47fa-8c9c-8bd182c9cac4 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:17:11.260896 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3debdc1-89d7-48bb-8dfb-7f6fd66abd3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:17:11.261719 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3debdc1-89d7-48bb-8dfb-7f6fd66abd3e admin admin] REQUEST_METHOD: `PUT` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:11.262066 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3debdc1-89d7-48bb-8dfb-7f6fd66abd3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:11.262502 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3debdc1-89d7-48bb-8dfb-7f6fd66abd3e admin admin] PATH_INFO: `/v3/projects/9f57110cd4574ca7b3eeecf20e1f0d1b/users/a9eb694acfd842369e2c398460187002/roles/49e47cd157d743f6905f1db07ec878a7` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:11.264472 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3debdc1-89d7-48bb-8dfb-7f6fd66abd3e admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f57110cd4574ca7b3eeecf20e1f0d1b, user_id=a9eb694acfd842369e2c398460187002, role_id=49e47cd157d743f6905f1db07ec878a7)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:17:11.267222 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3debdc1-89d7-48bb-8dfb-7f6fd66abd3e admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:17:11.285549 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 951/1906] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Aug 24 06:17:11 2022] PUT /identity/v3/projects/9f57110cd4574ca7b3eeecf20e1f0d1b/users/a9eb694acfd842369e2c398460187002/roles/49e47cd157d743f6905f1db07ec878a7 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:17:11.291925 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392920b3-aa74-4a62-af66-ec04da929e93 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:11.292218 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392920b3-aa74-4a62-af66-ec04da929e93 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:11.292522 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392920b3-aa74-4a62-af66-ec04da929e93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:11.324121 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-392920b3-aa74-4a62-af66-ec04da929e93 None None] MFA Rules not processed for user `a9eb694acfd842369e2c398460187002`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:17:11.324318 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-add27e72-b170-4775-b60e-2348a712bc3b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5053dda49bb14b28b9357947315170b6', 'request_id': 'req-add27e72-b170-4775-b60e-2348a712bc3b'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:17:11.324870 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-392920b3-aa74-4a62-af66-ec04da929e93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:17:11.330615 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-add27e72-b170-4775-b60e-2348a712bc3b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5053dda49bb14b28b9357947315170b6', 'request_id': 'req-add27e72-b170-4775-b60e-2348a712bc3b'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:17:11.339857 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-add27e72-b170-4775-b60e-2348a712bc3b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5053dda49bb14b28b9357947315170b6', 'request_id': 'req-add27e72-b170-4775-b60e-2348a712bc3b'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:17:11.420545 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 952/1907] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:17:11 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 06:17:11.434221 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392920b3-aa74-4a62-af66-ec04da929e93 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:17:11.438069 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-392920b3-aa74-4a62-af66-ec04da929e93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:17:11.438726 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-add27e72-b170-4775-b60e-2348a712bc3b admin admin] The token cache is being invalidate because project 5053dda49bb14b28b9357947315170b6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81851) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:17:11.438988 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-add27e72-b170-4775-b60e-2348a712bc3b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:17:11.441302 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-add27e72-b170-4775-b60e-2348a712bc3b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:17:11.443426 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 956/1908] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Aug 24 06:17:11 2022] DELETE /identity/v3/projects/5053dda49bb14b28b9357947315170b6 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:17:11.448412 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56260f25-5309-4a96-9c25-a5da2631e3cf None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:11.449100 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56260f25-5309-4a96-9c25-a5da2631e3cf None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:11.449375 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56260f25-5309-4a96-9c25-a5da2631e3cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:11.496968 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-56260f25-5309-4a96-9c25-a5da2631e3cf None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:17:11.498972 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-56260f25-5309-4a96-9c25-a5da2631e3cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:17:11.511627 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392920b3-aa74-4a62-af66-ec04da929e93 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:17:11.521700 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42404495-4d53-47d9-9c5c-4b86447df8c7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c4def55ac0e48b29836bc5c64bc7402', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:17:11.521700 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42404495-4d53-47d9-9c5c-4b86447df8c7 service cinder] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:11.521700 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42404495-4d53-47d9-9c5c-4b86447df8c7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:11.521700 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42404495-4d53-47d9-9c5c-4b86447df8c7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:11.522121 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-42404495-4d53-47d9-9c5c-4b86447df8c7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:17:11.544622 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42404495-4d53-47d9-9c5c-4b86447df8c7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:17:11.544622 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42404495-4d53-47d9-9c5c-4b86447df8c7 service cinder] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:17:11.557034 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 953/1909] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:17:11 2022] GET /identity/v3/auth/tokens => generated 3349 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:17:11.585885 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 957/1910] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:17:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:17:11.594120 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52674e74-054a-4b7d-995c-c64ed93f52b1 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:11.594597 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52674e74-054a-4b7d-995c-c64ed93f52b1 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:11.595028 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52674e74-054a-4b7d-995c-c64ed93f52b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:11.628814 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-52674e74-054a-4b7d-995c-c64ed93f52b1 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:17:11.630913 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-52674e74-054a-4b7d-995c-c64ed93f52b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:17:11.663499 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:17:11.671649 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 954/1911] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:17:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:17:11.671780 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56260f25-5309-4a96-9c25-a5da2631e3cf None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:17:11.701758 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56260f25-5309-4a96-9c25-a5da2631e3cf None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:17:11.713993 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105d7028-6fa2-43ea-96fb-19247a5cc935 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:17:11.714624 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105d7028-6fa2-43ea-96fb-19247a5cc935 admin admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:11.714904 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105d7028-6fa2-43ea-96fb-19247a5cc935 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:11.715171 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105d7028-6fa2-43ea-96fb-19247a5cc935 admin admin] PATH_INFO: `/v3/domains` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:11.715821 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105d7028-6fa2-43ea-96fb-19247a5cc935 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:17:11.717813 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105d7028-6fa2-43ea-96fb-19247a5cc935 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:17:11.719179 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105d7028-6fa2-43ea-96fb-19247a5cc935 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:17:11.733667 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 958/1912] 10.222.0.62 () {58 vars in 1153 bytes} [Wed Aug 24 06:17:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 06:17:11.737749 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52674e74-054a-4b7d-995c-c64ed93f52b1 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:17:11.747878 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52674e74-054a-4b7d-995c-c64ed93f52b1 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:17:11.754572 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f30ddcb-426e-4623-a96a-01ca8a250df9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:17:11.755192 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f30ddcb-426e-4623-a96a-01ca8a250df9 admin admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:11.755468 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f30ddcb-426e-4623-a96a-01ca8a250df9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:11.755737 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f30ddcb-426e-4623-a96a-01ca8a250df9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:11.756380 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f30ddcb-426e-4623-a96a-01ca8a250df9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:17:11.758337 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f30ddcb-426e-4623-a96a-01ca8a250df9 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:17:11.973423 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 955/1913] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Aug 24 06:17:11 2022] POST /identity/v3/projects => generated 377 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 06:17:11.974507 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105d7028-6fa2-43ea-96fb-19247a5cc935 admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:17:11.984326 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f30ddcb-426e-4623-a96a-01ca8a250df9 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:17:11.985270 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105d7028-6fa2-43ea-96fb-19247a5cc935 admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:17:11.986178 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-0f30ddcb-426e-4623-a96a-01ca8a250df9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:17:12.084087 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f86148-2b73-4d25-88cc-acbc44e291be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:17:12.084886 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f86148-2b73-4d25-88cc-acbc44e291be admin admin] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:12.085170 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f86148-2b73-4d25-88cc-acbc44e291be admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:12.085606 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f86148-2b73-4d25-88cc-acbc44e291be admin admin] PATH_INFO: `/v3/users` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:12.091818 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f86148-2b73-4d25-88cc-acbc44e291be admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:17:12.091818 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f86148-2b73-4d25-88cc-acbc44e291be admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:17:12.104236 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f30ddcb-426e-4623-a96a-01ca8a250df9 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:17:12.111238 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:17:12.119619 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa77fa3-4875-4653-a187-ca241f7785d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:17:12.120387 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa77fa3-4875-4653-a187-ca241f7785d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:12.120770 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa77fa3-4875-4653-a187-ca241f7785d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:12.121056 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa77fa3-4875-4653-a187-ca241f7785d9 admin admin] PATH_INFO: `/v3/projects/b16ede2290884266a93d6bf904c5d21f` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:12.122600 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa77fa3-4875-4653-a187-ca241f7785d9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b16ede2290884266a93d6bf904c5d21f)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:17:12.125504 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa77fa3-4875-4653-a187-ca241f7785d9 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:17:12.245513 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 959/1914] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Aug 24 06:17:11 2022] POST /identity/v3/users => generated 289 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:17:12.245513 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f86148-2b73-4d25-88cc-acbc44e291be admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:17:12.249314 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-91f86148-2b73-4d25-88cc-acbc44e291be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:17:12.271149 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-7fa77fa3-4875-4653-a187-ca241f7785d9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b16ede2290884266a93d6bf904c5d21f', 'request_id': 'req-7fa77fa3-4875-4653-a187-ca241f7785d9'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:17:12.281322 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-7fa77fa3-4875-4653-a187-ca241f7785d9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b16ede2290884266a93d6bf904c5d21f', 'request_id': 'req-7fa77fa3-4875-4653-a187-ca241f7785d9'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:17:12.289749 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-7fa77fa3-4875-4653-a187-ca241f7785d9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b16ede2290884266a93d6bf904c5d21f', 'request_id': 'req-7fa77fa3-4875-4653-a187-ca241f7785d9'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:17:12.347894 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f86148-2b73-4d25-88cc-acbc44e291be admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:17:12.387334 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-7fa77fa3-4875-4653-a187-ca241f7785d9 admin admin] The token cache is being invalidate because project b16ede2290884266a93d6bf904c5d21f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81852) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:17:12.388401 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-7fa77fa3-4875-4653-a187-ca241f7785d9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:17:12.390936 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-7fa77fa3-4875-4653-a187-ca241f7785d9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:17:12.396512 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 956/1915] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Aug 24 06:17:11 2022] DELETE /identity/v3/projects/b16ede2290884266a93d6bf904c5d21f => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:17:12.397853 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa77fa3-4875-4653-a187-ca241f7785d9 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:17:12.402104 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-7fa77fa3-4875-4653-a187-ca241f7785d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:17:12.413478 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3662892-8e23-43b8-9257-b58f567e5987 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c4def55ac0e48b29836bc5c64bc7402', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:17:12.414177 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3662892-8e23-43b8-9257-b58f567e5987 service cinder] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:12.414459 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3662892-8e23-43b8-9257-b58f567e5987 service cinder] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:12.415405 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3662892-8e23-43b8-9257-b58f567e5987 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:12.416704 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-e3662892-8e23-43b8-9257-b58f567e5987 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:17:12.515074 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa77fa3-4875-4653-a187-ca241f7785d9 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:17:12.517114 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3662892-8e23-43b8-9257-b58f567e5987 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:17:12.519305 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3662892-8e23-43b8-9257-b58f567e5987 service cinder] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:17:12.521893 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfd77b1-966e-49ab-9df2-27bfee64463d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:17:12.522517 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfd77b1-966e-49ab-9df2-27bfee64463d admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:12.522906 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfd77b1-966e-49ab-9df2-27bfee64463d admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:12.523176 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfd77b1-966e-49ab-9df2-27bfee64463d admin admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:12.525263 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfd77b1-966e-49ab-9df2-27bfee64463d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:17:12.530431 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfd77b1-966e-49ab-9df2-27bfee64463d admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:17:12.545259 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 960/1916] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:17:12 2022] GET /identity/v3/auth/tokens => generated 3286 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:17:12.547042 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e710b898-6e89-4d4f-a212-9deb749a37eb None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:12.548261 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e710b898-6e89-4d4f-a212-9deb749a37eb None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:12.553298 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e710b898-6e89-4d4f-a212-9deb749a37eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:12.578844 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-e710b898-6e89-4d4f-a212-9deb749a37eb None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:17:12.580273 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-e710b898-6e89-4d4f-a212-9deb749a37eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:17:12.594360 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 957/1917] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Aug 24 06:17:12 2022] GET /identity/v3/roles => generated 1420 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:17:12.596140 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfd77b1-966e-49ab-9df2-27bfee64463d admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:17:12.598079 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 961/1918] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:17:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:17:12.599532 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c4df68-2760-4d09-bd6e-c143baf849ac None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:12.599889 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c4df68-2760-4d09-bd6e-c143baf849ac None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:12.600175 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c4df68-2760-4d09-bd6e-c143baf849ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:12.628748 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfd77b1-966e-49ab-9df2-27bfee64463d admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:17:12.634093 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-85c4df68-2760-4d09-bd6e-c143baf849ac None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:17:12.635547 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-85c4df68-2760-4d09-bd6e-c143baf849ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:17:12.642706 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a2df42-fb1f-4c05-954e-9ab3caf642b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:17:12.644847 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a2df42-fb1f-4c05-954e-9ab3caf642b5 admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:12.645272 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a2df42-fb1f-4c05-954e-9ab3caf642b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:12.645556 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a2df42-fb1f-4c05-954e-9ab3caf642b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:12.649465 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a2df42-fb1f-4c05-954e-9ab3caf642b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:17:12.651468 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 962/1919] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:17:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:17:12.652438 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c4df68-2760-4d09-bd6e-c143baf849ac None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:17:12.653184 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a2df42-fb1f-4c05-954e-9ab3caf642b5 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:17:12.668712 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c4df68-2760-4d09-bd6e-c143baf849ac None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:17:12.678499 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 958/1920] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Aug 24 06:17:12 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:17:12.690014 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a2df42-fb1f-4c05-954e-9ab3caf642b5 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:17:12.694414 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72443116-ea97-4f5a-9f62-b0f881613052 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:17:12.698908 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72443116-ea97-4f5a-9f62-b0f881613052 admin admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:12.698908 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72443116-ea97-4f5a-9f62-b0f881613052 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:12.698908 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72443116-ea97-4f5a-9f62-b0f881613052 admin admin] PATH_INFO: `/v3/domains` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:12.698908 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72443116-ea97-4f5a-9f62-b0f881613052 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:17:12.698908 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72443116-ea97-4f5a-9f62-b0f881613052 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:17:12.698908 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72443116-ea97-4f5a-9f62-b0f881613052 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:17:12.717972 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a2df42-fb1f-4c05-954e-9ab3caf642b5 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:17:12.737501 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbf47dd-7dcd-4bb6-9de2-f4c1d97d2c18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:17:12.738442 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbf47dd-7dcd-4bb6-9de2-f4c1d97d2c18 admin admin] REQUEST_METHOD: `PUT` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:12.739995 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbf47dd-7dcd-4bb6-9de2-f4c1d97d2c18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:12.740391 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbf47dd-7dcd-4bb6-9de2-f4c1d97d2c18 admin admin] PATH_INFO: `/v3/projects/84a0d30512134d2ab6fcb752de4c51a2/users/83b007978b5f48519f8ccec3844977e5/roles/580aa7dbb38d4cff8f9d1bb8983f99ef` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:12.750017 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 963/1921] 10.222.0.62 () {58 vars in 1153 bytes} [Wed Aug 24 06:17:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:17:12.756650 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72443116-ea97-4f5a-9f62-b0f881613052 admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:17:12.788558 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72443116-ea97-4f5a-9f62-b0f881613052 admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:17:12.801809 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbf47dd-7dcd-4bb6-9de2-f4c1d97d2c18 admin admin] RBAC: Authorizing `identity:create_grant(project_id=84a0d30512134d2ab6fcb752de4c51a2, user_id=83b007978b5f48519f8ccec3844977e5, role_id=580aa7dbb38d4cff8f9d1bb8983f99ef)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:17:12.806849 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8312885d-f388-4927-b753-f6c69d3aa87c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:17:12.807447 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbf47dd-7dcd-4bb6-9de2-f4c1d97d2c18 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:17:12.807537 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8312885d-f388-4927-b753-f6c69d3aa87c admin admin] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:12.807732 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8312885d-f388-4927-b753-f6c69d3aa87c admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:12.808005 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8312885d-f388-4927-b753-f6c69d3aa87c admin admin] PATH_INFO: `/v3/projects` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:12.809426 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8312885d-f388-4927-b753-f6c69d3aa87c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:17:12.811392 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8312885d-f388-4927-b753-f6c69d3aa87c admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:17:12.925077 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 959/1922] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Aug 24 06:17:12 2022] PUT /identity/v3/projects/84a0d30512134d2ab6fcb752de4c51a2/users/83b007978b5f48519f8ccec3844977e5/roles/580aa7dbb38d4cff8f9d1bb8983f99ef => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:17:12.932519 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09749ce1-a21b-4280-8a6b-a8cddb0c4268 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:12.932810 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09749ce1-a21b-4280-8a6b-a8cddb0c4268 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:12.933080 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09749ce1-a21b-4280-8a6b-a8cddb0c4268 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:12.970707 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-09749ce1-a21b-4280-8a6b-a8cddb0c4268 None None] MFA Rules not processed for user `83b007978b5f48519f8ccec3844977e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:17:12.971931 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-09749ce1-a21b-4280-8a6b-a8cddb0c4268 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:17:13.002504 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 964/1923] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Aug 24 06:17:12 2022] POST /identity/v3/projects => generated 379 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:17:13.007207 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8312885d-f388-4927-b753-f6c69d3aa87c admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:17:13.020261 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:17:13.143518 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 960/1924] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:17:12 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 06:17:13.149812 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09749ce1-a21b-4280-8a6b-a8cddb0c4268 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:17:13.167595 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8312885d-f388-4927-b753-f6c69d3aa87c admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:17:13.182342 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84061022-b1f4-4f9b-b44a-de670c472a1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:17:13.182978 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84061022-b1f4-4f9b-b44a-de670c472a1c admin admin] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:13.186917 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84061022-b1f4-4f9b-b44a-de670c472a1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:13.186917 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84061022-b1f4-4f9b-b44a-de670c472a1c admin admin] PATH_INFO: `/v3/users` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:13.186917 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84061022-b1f4-4f9b-b44a-de670c472a1c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:17:13.186917 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84061022-b1f4-4f9b-b44a-de670c472a1c admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:17:13.257670 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09749ce1-a21b-4280-8a6b-a8cddb0c4268 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:17:13.271929 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102327fb-19a0-4c5a-b584-da731ec958dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:17:13.272627 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102327fb-19a0-4c5a-b584-da731ec958dd admin admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:13.272908 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102327fb-19a0-4c5a-b584-da731ec958dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:13.273178 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102327fb-19a0-4c5a-b584-da731ec958dd admin admin] PATH_INFO: `/v3/projects` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:13.273809 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102327fb-19a0-4c5a-b584-da731ec958dd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:17:13.276354 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102327fb-19a0-4c5a-b584-da731ec958dd admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:17:13.328126 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 965/1925] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Aug 24 06:17:13 2022] POST /identity/v3/users => generated 290 bytes in 322 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 06:17:13.333452 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84061022-b1f4-4f9b-b44a-de670c472a1c admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:17:13.343839 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84061022-b1f4-4f9b-b44a-de670c472a1c admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:17:13.363114 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:17:13.365999 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce65576e-0d49-4a0a-b14a-f40e36ef74bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:17:13.367128 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce65576e-0d49-4a0a-b14a-f40e36ef74bb admin admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:13.367268 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce65576e-0d49-4a0a-b14a-f40e36ef74bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:13.367602 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce65576e-0d49-4a0a-b14a-f40e36ef74bb admin admin] PATH_INFO: `/v3/roles` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:13.368901 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce65576e-0d49-4a0a-b14a-f40e36ef74bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:17:13.370599 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce65576e-0d49-4a0a-b14a-f40e36ef74bb admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:17:13.392013 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 966/1926] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Aug 24 06:17:13 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 06:17:13.399259 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce65576e-0d49-4a0a-b14a-f40e36ef74bb admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:17:13.415222 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 961/1927] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Aug 24 06:17:13 2022] POST /identity/v3/projects => generated 377 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:17:13.417996 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102327fb-19a0-4c5a-b584-da731ec958dd admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:17:13.421134 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce65576e-0d49-4a0a-b14a-f40e36ef74bb admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:17:13.534569 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102327fb-19a0-4c5a-b584-da731ec958dd admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:17:13.545054 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa11c127-035e-464c-8062-c788643859cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:17:13.545871 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa11c127-035e-464c-8062-c788643859cf admin admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:13.546157 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa11c127-035e-464c-8062-c788643859cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:13.546437 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa11c127-035e-464c-8062-c788643859cf admin admin] PATH_INFO: `/v3/users` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:13.547078 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa11c127-035e-464c-8062-c788643859cf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:17:13.549335 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa11c127-035e-464c-8062-c788643859cf admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:17:13.589313 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07482f7-e04f-4c64-b163-be8b20100017 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:17:13.593131 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07482f7-e04f-4c64-b163-be8b20100017 admin admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:13.593833 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07482f7-e04f-4c64-b163-be8b20100017 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:13.594221 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07482f7-e04f-4c64-b163-be8b20100017 admin admin] PATH_INFO: `/v3/roles` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:13.596329 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07482f7-e04f-4c64-b163-be8b20100017 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:17:13.600465 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07482f7-e04f-4c64-b163-be8b20100017 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:17:13.628598 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 967/1928] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Aug 24 06:17:13 2022] GET /identity/v3/roles => generated 1420 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:17:13.634245 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07482f7-e04f-4c64-b163-be8b20100017 admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:17:13.647095 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07482f7-e04f-4c64-b163-be8b20100017 admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:17:13.652345 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc71b7e-7061-4382-a4a1-e32838ee3f78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:17:13.653158 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc71b7e-7061-4382-a4a1-e32838ee3f78 admin admin] REQUEST_METHOD: `PUT` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:13.653506 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc71b7e-7061-4382-a4a1-e32838ee3f78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:13.653790 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc71b7e-7061-4382-a4a1-e32838ee3f78 admin admin] PATH_INFO: `/v3/projects/183f907b77d84b4e89f32df021315633/users/a9cea11d6d244d0da7129f658ac6655e/roles/580aa7dbb38d4cff8f9d1bb8983f99ef` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:13.676453 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc71b7e-7061-4382-a4a1-e32838ee3f78 admin admin] RBAC: Authorizing `identity:create_grant(project_id=183f907b77d84b4e89f32df021315633, user_id=a9cea11d6d244d0da7129f658ac6655e, role_id=580aa7dbb38d4cff8f9d1bb8983f99ef)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:17:13.679210 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc71b7e-7061-4382-a4a1-e32838ee3f78 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:17:14.475569 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 962/1929] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Aug 24 06:17:13 2022] POST /identity/v3/users => generated 295 bytes in 1058 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 06:17:14.477257 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 968/1930] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Aug 24 06:17:13 2022] PUT /identity/v3/projects/183f907b77d84b4e89f32df021315633/users/a9cea11d6d244d0da7129f658ac6655e/roles/580aa7dbb38d4cff8f9d1bb8983f99ef => generated 0 bytes in 843 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:17:14.483409 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa11c127-035e-464c-8062-c788643859cf admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:17:14.488522 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b111ab-d26e-4901-979e-993a52baf04e None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:14.488522 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b111ab-d26e-4901-979e-993a52baf04e None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:14.488828 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b111ab-d26e-4901-979e-993a52baf04e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:14.505617 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa11c127-035e-464c-8062-c788643859cf admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:17:14.519998 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-b2b111ab-d26e-4901-979e-993a52baf04e None None] MFA Rules not processed for user `a9cea11d6d244d0da7129f658ac6655e`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:17:14.521413 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-b2b111ab-d26e-4901-979e-993a52baf04e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:17:14.572296 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878c7cdd-737a-4be6-8cb9-9c3b38c85c78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:17:14.573076 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878c7cdd-737a-4be6-8cb9-9c3b38c85c78 admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:14.573459 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878c7cdd-737a-4be6-8cb9-9c3b38c85c78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:14.573877 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878c7cdd-737a-4be6-8cb9-9c3b38c85c78 admin admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:14.574712 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878c7cdd-737a-4be6-8cb9-9c3b38c85c78 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:17:14.577766 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878c7cdd-737a-4be6-8cb9-9c3b38c85c78 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:17:14.590789 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 963/1931] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Aug 24 06:17:14 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:17:14.595478 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878c7cdd-737a-4be6-8cb9-9c3b38c85c78 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:17:14.608460 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878c7cdd-737a-4be6-8cb9-9c3b38c85c78 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:17:14.609981 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 969/1932] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:17:14 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 06:17:14.614625 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b111ab-d26e-4901-979e-993a52baf04e None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:17:14.619389 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7871f7ea-927f-46a2-b914-0d70511867c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:17:14.620230 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7871f7ea-927f-46a2-b914-0d70511867c0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:14.620682 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7871f7ea-927f-46a2-b914-0d70511867c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:14.622212 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7871f7ea-927f-46a2-b914-0d70511867c0 admin admin] PATH_INFO: `/v3/projects/5562f66712d34de5ba7e35b6d2955276/users/ba8f435de9d749aa9243f7aaba13d192/roles/49e47cd157d743f6905f1db07ec878a7` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:14.631938 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b111ab-d26e-4901-979e-993a52baf04e None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:17:14.638570 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a2a283-e276-431c-8a42-9d31121c62cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:17:14.639346 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a2a283-e276-431c-8a42-9d31121c62cf admin admin] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:14.639633 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a2a283-e276-431c-8a42-9d31121c62cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:14.639910 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a2a283-e276-431c-8a42-9d31121c62cf admin admin] PATH_INFO: `/v3/projects` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:14.643213 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a2a283-e276-431c-8a42-9d31121c62cf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:17:14.643213 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a2a283-e276-431c-8a42-9d31121c62cf admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:17:14.650295 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7871f7ea-927f-46a2-b914-0d70511867c0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5562f66712d34de5ba7e35b6d2955276, user_id=ba8f435de9d749aa9243f7aaba13d192, role_id=49e47cd157d743f6905f1db07ec878a7)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:17:14.663749 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7871f7ea-927f-46a2-b914-0d70511867c0 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:17:14.766566 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 970/1933] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Aug 24 06:17:14 2022] POST /identity/v3/projects => generated 381 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:17:14.772440 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a2a283-e276-431c-8a42-9d31121c62cf admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:17:14.803005 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 964/1934] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Aug 24 06:17:14 2022] PUT /identity/v3/projects/5562f66712d34de5ba7e35b6d2955276/users/ba8f435de9d749aa9243f7aaba13d192/roles/49e47cd157d743f6905f1db07ec878a7 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 06:17:14.812748 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7871f7ea-927f-46a2-b914-0d70511867c0 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:17:14.892590 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a2a283-e276-431c-8a42-9d31121c62cf admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:17:14.974556 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7871f7ea-927f-46a2-b914-0d70511867c0 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:17:14.986324 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3732fd2-19fd-4abc-bdba-6c212ddac4db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:17:14.987266 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3732fd2-19fd-4abc-bdba-6c212ddac4db admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:14.988118 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3732fd2-19fd-4abc-bdba-6c212ddac4db admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:14.988570 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3732fd2-19fd-4abc-bdba-6c212ddac4db admin admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:14.989214 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3732fd2-19fd-4abc-bdba-6c212ddac4db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:17:15.002800 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3732fd2-19fd-4abc-bdba-6c212ddac4db admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:17:15.019339 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 965/1935] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Aug 24 06:17:14 2022] GET /identity/v3/roles => generated 1420 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:17:15.024152 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3732fd2-19fd-4abc-bdba-6c212ddac4db admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:17:15.036895 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e06f74-e835-4a79-8272-555378526832 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:17:15.037661 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3732fd2-19fd-4abc-bdba-6c212ddac4db admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:17:15.043002 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e06f74-e835-4a79-8272-555378526832 admin admin] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:15.043002 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e06f74-e835-4a79-8272-555378526832 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:15.043002 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e06f74-e835-4a79-8272-555378526832 admin admin] PATH_INFO: `/v3/users` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:15.043002 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e06f74-e835-4a79-8272-555378526832 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:17:15.043002 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e06f74-e835-4a79-8272-555378526832 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:17:15.045108 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a235176-7369-4970-ac3c-8ced597891c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:17:15.045894 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a235176-7369-4970-ac3c-8ced597891c3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:15.046290 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a235176-7369-4970-ac3c-8ced597891c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:15.046635 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a235176-7369-4970-ac3c-8ced597891c3 admin admin] PATH_INFO: `/v3/projects/5562f66712d34de5ba7e35b6d2955276/users/ba8f435de9d749aa9243f7aaba13d192/roles/49e47cd157d743f6905f1db07ec878a7` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:15.049694 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a235176-7369-4970-ac3c-8ced597891c3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5562f66712d34de5ba7e35b6d2955276, user_id=ba8f435de9d749aa9243f7aaba13d192, role_id=49e47cd157d743f6905f1db07ec878a7)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:17:15.057732 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a235176-7369-4970-ac3c-8ced597891c3 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:17:15.078253 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 966/1936] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Aug 24 06:17:15 2022] PUT /identity/v3/projects/5562f66712d34de5ba7e35b6d2955276/users/ba8f435de9d749aa9243f7aaba13d192/roles/49e47cd157d743f6905f1db07ec878a7 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:17:15.091865 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034be30c-30fb-4220-a662-e76a7b74d0be None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:15.092816 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034be30c-30fb-4220-a662-e76a7b74d0be None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:15.093239 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034be30c-30fb-4220-a662-e76a7b74d0be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:15.127202 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-034be30c-30fb-4220-a662-e76a7b74d0be None None] MFA Rules not processed for user `ba8f435de9d749aa9243f7aaba13d192`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:17:15.129474 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-034be30c-30fb-4220-a662-e76a7b74d0be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:17:15.163931 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 971/1937] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Aug 24 06:17:14 2022] POST /identity/v3/users => generated 297 bytes in 392 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 06:17:15.168421 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e06f74-e835-4a79-8272-555378526832 admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:17:15.199063 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e06f74-e835-4a79-8272-555378526832 admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:17:15.273785 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45e6087-7e1b-4571-b451-17a726496210 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:17:15.274501 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45e6087-7e1b-4571-b451-17a726496210 admin admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:15.274684 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45e6087-7e1b-4571-b451-17a726496210 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:15.274939 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45e6087-7e1b-4571-b451-17a726496210 admin admin] PATH_INFO: `/v3/roles` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:15.275902 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45e6087-7e1b-4571-b451-17a726496210 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:17:15.277537 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45e6087-7e1b-4571-b451-17a726496210 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:17:15.288631 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 967/1938] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:17:15 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 06:17:15.297873 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 972/1939] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Aug 24 06:17:15 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 06:17:15.298986 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034be30c-30fb-4220-a662-e76a7b74d0be None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:17:15.303430 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45e6087-7e1b-4571-b451-17a726496210 admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:17:15.308936 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-e45e6087-7e1b-4571-b451-17a726496210 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:17:15.317528 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034be30c-30fb-4220-a662-e76a7b74d0be None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:17:15.327627 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a55254-6b8c-4110-af6e-b2049fd07826 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:17:15.328317 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a55254-6b8c-4110-af6e-b2049fd07826 admin admin] REQUEST_METHOD: `PUT` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:15.328628 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a55254-6b8c-4110-af6e-b2049fd07826 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:15.328907 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a55254-6b8c-4110-af6e-b2049fd07826 admin admin] PATH_INFO: `/v3/projects/d4d9ccdd0928458894a127cb4ef2af5c/users/0f252d1d6e124be29a369dc557bbbfa3/roles/49e47cd157d743f6905f1db07ec878a7` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:15.366512 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a55254-6b8c-4110-af6e-b2049fd07826 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d4d9ccdd0928458894a127cb4ef2af5c, user_id=0f252d1d6e124be29a369dc557bbbfa3, role_id=49e47cd157d743f6905f1db07ec878a7)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:17:15.369753 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a55254-6b8c-4110-af6e-b2049fd07826 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:17:15.405106 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45e6087-7e1b-4571-b451-17a726496210 admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:17:15.410372 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e39b20-1da1-45d6-a033-8e8816cbe818 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c4def55ac0e48b29836bc5c64bc7402', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:17:15.411450 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e39b20-1da1-45d6-a033-8e8816cbe818 service cinder] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:15.411825 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e39b20-1da1-45d6-a033-8e8816cbe818 service cinder] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:15.412279 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e39b20-1da1-45d6-a033-8e8816cbe818 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:15.483733 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 968/1940] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Aug 24 06:17:15 2022] PUT /identity/v3/projects/d4d9ccdd0928458894a127cb4ef2af5c/users/0f252d1d6e124be29a369dc557bbbfa3/roles/49e47cd157d743f6905f1db07ec878a7 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:17:15.484901 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a55254-6b8c-4110-af6e-b2049fd07826 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:17:15.511602 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e39b20-1da1-45d6-a033-8e8816cbe818 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:17:15.513042 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e39b20-1da1-45d6-a033-8e8816cbe818 service cinder] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:17:15.524168 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 973/1941] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:17:15 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 06:17:15.587845 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a55254-6b8c-4110-af6e-b2049fd07826 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:17:15.602097 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5b49a8-b3cf-4a44-bfb1-6149be14e30f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:17:15.602766 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5b49a8-b3cf-4a44-bfb1-6149be14e30f admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:15.603516 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5b49a8-b3cf-4a44-bfb1-6149be14e30f admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:15.603893 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5b49a8-b3cf-4a44-bfb1-6149be14e30f admin admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:15.605119 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5b49a8-b3cf-4a44-bfb1-6149be14e30f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:17:15.607668 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5b49a8-b3cf-4a44-bfb1-6149be14e30f admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:17:15.642809 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 969/1942] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Aug 24 06:17:15 2022] GET /identity/v3/roles => generated 1420 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 06:17:15.644092 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e39b20-1da1-45d6-a033-8e8816cbe818 service cinder] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:17:15.659963 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e39b20-1da1-45d6-a033-8e8816cbe818 service cinder] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:17:15.677400 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c181ea-e53e-4f4d-a3af-446a506692fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:17:15.678338 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c181ea-e53e-4f4d-a3af-446a506692fd admin admin] REQUEST_METHOD: `PUT` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:15.678620 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c181ea-e53e-4f4d-a3af-446a506692fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:15.678893 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c181ea-e53e-4f4d-a3af-446a506692fd admin admin] PATH_INFO: `/v3/projects/d4d9ccdd0928458894a127cb4ef2af5c/users/0f252d1d6e124be29a369dc557bbbfa3/roles/49e47cd157d743f6905f1db07ec878a7` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:15.685695 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c181ea-e53e-4f4d-a3af-446a506692fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=d4d9ccdd0928458894a127cb4ef2af5c, user_id=0f252d1d6e124be29a369dc557bbbfa3, role_id=49e47cd157d743f6905f1db07ec878a7)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:17:15.694044 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c181ea-e53e-4f4d-a3af-446a506692fd admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:17:15.700050 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d17540-1132-4f16-a855-af9ecf1cada8 None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:15.700427 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d17540-1132-4f16-a855-af9ecf1cada8 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:15.700874 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d17540-1132-4f16-a855-af9ecf1cada8 None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:15.704859 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 970/1943] 10.222.0.62 () {54 vars in 921 bytes} [Wed Aug 24 06:17:15 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:17:15.727885 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0585c1a5-35f5-4d28-b630-ea8320936cf4 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:15.728256 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0585c1a5-35f5-4d28-b630-ea8320936cf4 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:15.728594 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0585c1a5-35f5-4d28-b630-ea8320936cf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:15.743009 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:17:15.744652 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 974/1944] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Aug 24 06:17:15 2022] PUT /identity/v3/projects/d4d9ccdd0928458894a127cb4ef2af5c/users/0f252d1d6e124be29a369dc557bbbfa3/roles/49e47cd157d743f6905f1db07ec878a7 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:17:15.754508 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79ef20e-968a-4358-b29e-0ce470267314 None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:15.754787 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79ef20e-968a-4358-b29e-0ce470267314 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:15.755331 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79ef20e-968a-4358-b29e-0ce470267314 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:15.785993 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-0585c1a5-35f5-4d28-b630-ea8320936cf4 None None] MFA Rules not processed for user `fb6d3743340241f98cacb9e0d063298c`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:17:15.792220 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-0585c1a5-35f5-4d28-b630-ea8320936cf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:17:15.850001 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-c79ef20e-968a-4358-b29e-0ce470267314 None None] MFA Rules not processed for user `0f252d1d6e124be29a369dc557bbbfa3`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:17:15.854738 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-c79ef20e-968a-4358-b29e-0ce470267314 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:17:15.970421 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 971/1945] 10.222.0.62 () {58 vars in 1018 bytes} [Wed Aug 24 06:17:15 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:17:16.026215 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 975/1946] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:17:15 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 268 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:17:16.126473 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0585c1a5-35f5-4d28-b630-ea8320936cf4 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:17:16.128709 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-0585c1a5-35f5-4d28-b630-ea8320936cf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:17:16.135359 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:17:16.149005 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0585c1a5-35f5-4d28-b630-ea8320936cf4 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:17:16.155806 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ab46f3-5a55-4d50-915b-6f75f9c58e5a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb6d3743340241f98cacb9e0d063298c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:17:16.156770 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ab46f3-5a55-4d50-915b-6f75f9c58e5a service nova] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:16.157314 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ab46f3-5a55-4d50-915b-6f75f9c58e5a service nova] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:16.157858 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ab46f3-5a55-4d50-915b-6f75f9c58e5a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:16.172309 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ab46f3-5a55-4d50-915b-6f75f9c58e5a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:17:16.175430 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ab46f3-5a55-4d50-915b-6f75f9c58e5a service nova] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:17:16.200299 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 972/1947] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:17:16 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 74 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:17:16.580587 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79ef20e-968a-4358-b29e-0ce470267314 None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:17:16.826380 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79ef20e-968a-4358-b29e-0ce470267314 None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:17:16.831870 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b56e11-8859-4c1d-9370-3d167285196e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c4def55ac0e48b29836bc5c64bc7402', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:17:16.832836 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b56e11-8859-4c1d-9370-3d167285196e service cinder] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:16.833315 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b56e11-8859-4c1d-9370-3d167285196e service cinder] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:16.833750 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b56e11-8859-4c1d-9370-3d167285196e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:16.848219 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b56e11-8859-4c1d-9370-3d167285196e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:17:16.851388 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b56e11-8859-4c1d-9370-3d167285196e service cinder] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:17:16.864550 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 976/1948] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:17:16 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:17:17.909338 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:17:18.074784 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ab46f3-5a55-4d50-915b-6f75f9c58e5a service nova] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:17:18.090748 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ab46f3-5a55-4d50-915b-6f75f9c58e5a service nova] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:17:18.094260 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925fe440-8c16-4c4d-91e8-cf59f7f852b3 tempest-VolumeQuotaClassesTest-1003191381 tempest-VolumeQuotaClassesTest-1003191381-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f252d1d6e124be29a369dc557bbbfa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4d9ccdd0928458894a127cb4ef2af5c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:17:18.094837 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925fe440-8c16-4c4d-91e8-cf59f7f852b3 tempest-VolumeQuotaClassesTest-1003191381 tempest-VolumeQuotaClassesTest-1003191381-project-admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:18.095117 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925fe440-8c16-4c4d-91e8-cf59f7f852b3 tempest-VolumeQuotaClassesTest-1003191381 tempest-VolumeQuotaClassesTest-1003191381-project-admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:18.095585 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925fe440-8c16-4c4d-91e8-cf59f7f852b3 tempest-VolumeQuotaClassesTest-1003191381 tempest-VolumeQuotaClassesTest-1003191381-project-admin] PATH_INFO: `/v3/domains` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:18.096333 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925fe440-8c16-4c4d-91e8-cf59f7f852b3 tempest-VolumeQuotaClassesTest-1003191381 tempest-VolumeQuotaClassesTest-1003191381-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:17:18.096659 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925fe440-8c16-4c4d-91e8-cf59f7f852b3 tempest-VolumeQuotaClassesTest-1003191381 tempest-VolumeQuotaClassesTest-1003191381-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:17:18.098278 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925fe440-8c16-4c4d-91e8-cf59f7f852b3 tempest-VolumeQuotaClassesTest-1003191381 tempest-VolumeQuotaClassesTest-1003191381-project-admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:17:18.111678 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 973/1949] 10.222.0.62 () {58 vars in 1153 bytes} [Wed Aug 24 06:17:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:17:18.116132 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b56e11-8859-4c1d-9370-3d167285196e service cinder] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:17:18.126021 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b56e11-8859-4c1d-9370-3d167285196e service cinder] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:17:18.133535 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40da46bc-6caa-4f1d-8ee1-ea2bc0fcf5e4 tempest-VolumeQuotaClassesTest-1003191381 tempest-VolumeQuotaClassesTest-1003191381-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f252d1d6e124be29a369dc557bbbfa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4d9ccdd0928458894a127cb4ef2af5c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:17:18.134170 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40da46bc-6caa-4f1d-8ee1-ea2bc0fcf5e4 tempest-VolumeQuotaClassesTest-1003191381 tempest-VolumeQuotaClassesTest-1003191381-project-admin] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:18.134452 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40da46bc-6caa-4f1d-8ee1-ea2bc0fcf5e4 tempest-VolumeQuotaClassesTest-1003191381 tempest-VolumeQuotaClassesTest-1003191381-project-admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:18.134728 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40da46bc-6caa-4f1d-8ee1-ea2bc0fcf5e4 tempest-VolumeQuotaClassesTest-1003191381 tempest-VolumeQuotaClassesTest-1003191381-project-admin] PATH_INFO: `/v3/projects` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:18.135570 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40da46bc-6caa-4f1d-8ee1-ea2bc0fcf5e4 tempest-VolumeQuotaClassesTest-1003191381 tempest-VolumeQuotaClassesTest-1003191381-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:17:18.137562 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40da46bc-6caa-4f1d-8ee1-ea2bc0fcf5e4 tempest-VolumeQuotaClassesTest-1003191381 tempest-VolumeQuotaClassesTest-1003191381-project-admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:17:18.220527 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 977/1950] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Aug 24 06:17:18 2022] POST /identity/v3/projects => generated 352 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:17:18.224574 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925fe440-8c16-4c4d-91e8-cf59f7f852b3 tempest-VolumeQuotaClassesTest-1003191381 tempest-VolumeQuotaClassesTest-1003191381-project-admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:17:18.238751 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925fe440-8c16-4c4d-91e8-cf59f7f852b3 tempest-VolumeQuotaClassesTest-1003191381 tempest-VolumeQuotaClassesTest-1003191381-project-admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:17:18.280756 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:17:18.317883 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03581c87-122f-4435-bf7e-b5b05924ea34 None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:18.319892 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03581c87-122f-4435-bf7e-b5b05924ea34 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:18.319892 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03581c87-122f-4435-bf7e-b5b05924ea34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:18.322346 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3884b8d4-fb97-4c4a-adbe-27971da855fe tempest-VolumeQuotaClassesTest-1003191381 tempest-VolumeQuotaClassesTest-1003191381-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f252d1d6e124be29a369dc557bbbfa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4d9ccdd0928458894a127cb4ef2af5c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:17:18.324484 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3884b8d4-fb97-4c4a-adbe-27971da855fe tempest-VolumeQuotaClassesTest-1003191381 tempest-VolumeQuotaClassesTest-1003191381-project-admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:18.324830 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3884b8d4-fb97-4c4a-adbe-27971da855fe tempest-VolumeQuotaClassesTest-1003191381 tempest-VolumeQuotaClassesTest-1003191381-project-admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:18.326245 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3884b8d4-fb97-4c4a-adbe-27971da855fe tempest-VolumeQuotaClassesTest-1003191381 tempest-VolumeQuotaClassesTest-1003191381-project-admin] PATH_INFO: `/v3/domains` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:18.327571 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3884b8d4-fb97-4c4a-adbe-27971da855fe tempest-VolumeQuotaClassesTest-1003191381 tempest-VolumeQuotaClassesTest-1003191381-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:17:18.327888 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3884b8d4-fb97-4c4a-adbe-27971da855fe tempest-VolumeQuotaClassesTest-1003191381 tempest-VolumeQuotaClassesTest-1003191381-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:17:18.329580 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3884b8d4-fb97-4c4a-adbe-27971da855fe tempest-VolumeQuotaClassesTest-1003191381 tempest-VolumeQuotaClassesTest-1003191381-project-admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:17:18.349423 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 974/1951] 10.222.0.62 () {58 vars in 1153 bytes} [Wed Aug 24 06:17:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:17:18.361178 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-03581c87-122f-4435-bf7e-b5b05924ea34 None None] MFA Rules not processed for user `d3a03b5f82724f37a2134554d42b2925`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:17:18.363532 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-03581c87-122f-4435-bf7e-b5b05924ea34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:17:18.398845 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3884b8d4-fb97-4c4a-adbe-27971da855fe tempest-VolumeQuotaClassesTest-1003191381 tempest-VolumeQuotaClassesTest-1003191381-project-admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:17:18.409332 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3884b8d4-fb97-4c4a-adbe-27971da855fe tempest-VolumeQuotaClassesTest-1003191381 tempest-VolumeQuotaClassesTest-1003191381-project-admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:17:18.421425 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93bb18e-6de1-4b36-8db6-157d0c981142 tempest-VolumeQuotaClassesTest-1003191381 tempest-VolumeQuotaClassesTest-1003191381-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f252d1d6e124be29a369dc557bbbfa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4d9ccdd0928458894a127cb4ef2af5c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:17:18.421425 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93bb18e-6de1-4b36-8db6-157d0c981142 tempest-VolumeQuotaClassesTest-1003191381 tempest-VolumeQuotaClassesTest-1003191381-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:18.421425 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93bb18e-6de1-4b36-8db6-157d0c981142 tempest-VolumeQuotaClassesTest-1003191381 tempest-VolumeQuotaClassesTest-1003191381-project-admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:18.421762 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93bb18e-6de1-4b36-8db6-157d0c981142 tempest-VolumeQuotaClassesTest-1003191381 tempest-VolumeQuotaClassesTest-1003191381-project-admin] PATH_INFO: `/v3/projects/98003f79480445f58f03a73bd5b525fb` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:18.421762 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93bb18e-6de1-4b36-8db6-157d0c981142 tempest-VolumeQuotaClassesTest-1003191381 tempest-VolumeQuotaClassesTest-1003191381-project-admin] RBAC: Authorizing `identity:delete_project(project_id=98003f79480445f58f03a73bd5b525fb)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:17:18.421762 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93bb18e-6de1-4b36-8db6-157d0c981142 tempest-VolumeQuotaClassesTest-1003191381 tempest-VolumeQuotaClassesTest-1003191381-project-admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:17:18.463665 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 978/1952] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:17:18 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 06:17:18.468109 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03581c87-122f-4435-bf7e-b5b05924ea34 None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:17:18.469637 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-03581c87-122f-4435-bf7e-b5b05924ea34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:17:18.538299 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03581c87-122f-4435-bf7e-b5b05924ea34 None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:17:18.543530 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8b34a7-0f3b-4c44-b830-5a3806211770 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:17:18.546984 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8b34a7-0f3b-4c44-b830-5a3806211770 admin admin] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:18.546984 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8b34a7-0f3b-4c44-b830-5a3806211770 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:18.546984 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8b34a7-0f3b-4c44-b830-5a3806211770 admin admin] PATH_INFO: `/v3/projects` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:18.546984 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8b34a7-0f3b-4c44-b830-5a3806211770 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:17:18.547587 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8b34a7-0f3b-4c44-b830-5a3806211770 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:17:18.559205 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-c93bb18e-6de1-4b36-8db6-157d0c981142 tempest-VolumeQuotaClassesTest-1003191381 tempest-VolumeQuotaClassesTest-1003191381-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '98003f79480445f58f03a73bd5b525fb', 'request_id': 'req-c93bb18e-6de1-4b36-8db6-157d0c981142'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:17:18.568894 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-c93bb18e-6de1-4b36-8db6-157d0c981142 tempest-VolumeQuotaClassesTest-1003191381 tempest-VolumeQuotaClassesTest-1003191381-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '98003f79480445f58f03a73bd5b525fb', 'request_id': 'req-c93bb18e-6de1-4b36-8db6-157d0c981142'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:17:18.576879 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-c93bb18e-6de1-4b36-8db6-157d0c981142 tempest-VolumeQuotaClassesTest-1003191381 tempest-VolumeQuotaClassesTest-1003191381-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '98003f79480445f58f03a73bd5b525fb', 'request_id': 'req-c93bb18e-6de1-4b36-8db6-157d0c981142'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:17:18.670160 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 979/1953] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Aug 24 06:17:18 2022] POST /identity/v3/projects => generated 383 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 06:17:18.674488 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8b34a7-0f3b-4c44-b830-5a3806211770 admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:17:18.692449 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-c93bb18e-6de1-4b36-8db6-157d0c981142 tempest-VolumeQuotaClassesTest-1003191381 tempest-VolumeQuotaClassesTest-1003191381-project-admin] The token cache is being invalidate because project 98003f79480445f58f03a73bd5b525fb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81852) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:17:18.692833 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-c93bb18e-6de1-4b36-8db6-157d0c981142 tempest-VolumeQuotaClassesTest-1003191381 tempest-VolumeQuotaClassesTest-1003191381-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:17:18.693591 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-c93bb18e-6de1-4b36-8db6-157d0c981142 tempest-VolumeQuotaClassesTest-1003191381 tempest-VolumeQuotaClassesTest-1003191381-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:17:18.695509 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 975/1954] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Aug 24 06:17:18 2022] DELETE /identity/v3/projects/98003f79480445f58f03a73bd5b525fb => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 06:17:18.789204 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8b34a7-0f3b-4c44-b830-5a3806211770 admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:17:18.798530 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9b3b26-ad40-4163-b030-e459b4669664 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:17:18.799223 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9b3b26-ad40-4163-b030-e459b4669664 admin admin] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:18.799560 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9b3b26-ad40-4163-b030-e459b4669664 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:18.800350 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9b3b26-ad40-4163-b030-e459b4669664 admin admin] PATH_INFO: `/v3/users` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:18.801652 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9b3b26-ad40-4163-b030-e459b4669664 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:17:18.802963 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9b3b26-ad40-4163-b030-e459b4669664 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:17:18.944528 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93bb18e-6de1-4b36-8db6-157d0c981142 tempest-VolumeQuotaClassesTest-1003191381 tempest-VolumeQuotaClassesTest-1003191381-project-admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:17:18.944528 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-c93bb18e-6de1-4b36-8db6-157d0c981142 tempest-VolumeQuotaClassesTest-1003191381 tempest-VolumeQuotaClassesTest-1003191381-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:17:18.944929 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 980/1955] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Aug 24 06:17:18 2022] POST /identity/v3/users => generated 298 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 06:17:19.005764 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93bb18e-6de1-4b36-8db6-157d0c981142 tempest-VolumeQuotaClassesTest-1003191381 tempest-VolumeQuotaClassesTest-1003191381-project-admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:17:19.010582 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e482722d-430f-41bb-bc15-bef77f7478c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:17:19.011211 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e482722d-430f-41bb-bc15-bef77f7478c8 admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:19.011440 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e482722d-430f-41bb-bc15-bef77f7478c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:19.011711 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e482722d-430f-41bb-bc15-bef77f7478c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:19.012355 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e482722d-430f-41bb-bc15-bef77f7478c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:17:19.013994 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e482722d-430f-41bb-bc15-bef77f7478c8 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:17:19.027172 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 976/1956] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Aug 24 06:17:18 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 06:17:19.031443 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9b3b26-ad40-4163-b030-e459b4669664 admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:17:19.041988 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9b3b26-ad40-4163-b030-e459b4669664 admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:17:19.046973 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e52f3b5-eb4c-43cc-a993-8079becfcc3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:17:19.047636 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e52f3b5-eb4c-43cc-a993-8079becfcc3f admin admin] REQUEST_METHOD: `PUT` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:19.047917 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e52f3b5-eb4c-43cc-a993-8079becfcc3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:19.048475 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e52f3b5-eb4c-43cc-a993-8079becfcc3f admin admin] PATH_INFO: `/v3/projects/7d37379ce6c24acf83bd68e06fca20f8/users/d2b44c8575f54d6a82f40b9c3b70c38e/roles/49e47cd157d743f6905f1db07ec878a7` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:19.065058 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e52f3b5-eb4c-43cc-a993-8079becfcc3f admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d37379ce6c24acf83bd68e06fca20f8, user_id=d2b44c8575f54d6a82f40b9c3b70c38e, role_id=49e47cd157d743f6905f1db07ec878a7)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:17:19.071862 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e52f3b5-eb4c-43cc-a993-8079becfcc3f admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:17:19.145283 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 981/1957] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Aug 24 06:17:19 2022] PUT /identity/v3/projects/7d37379ce6c24acf83bd68e06fca20f8/users/d2b44c8575f54d6a82f40b9c3b70c38e/roles/49e47cd157d743f6905f1db07ec878a7 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:17:19.149241 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e482722d-430f-41bb-bc15-bef77f7478c8 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:17:19.159946 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e482722d-430f-41bb-bc15-bef77f7478c8 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:17:19.217113 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189a1486-a2fe-47cc-8fbc-af7afda5adbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:17:19.217724 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189a1486-a2fe-47cc-8fbc-af7afda5adbe admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:19.217999 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189a1486-a2fe-47cc-8fbc-af7afda5adbe admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:19.218268 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189a1486-a2fe-47cc-8fbc-af7afda5adbe admin admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:19.218906 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189a1486-a2fe-47cc-8fbc-af7afda5adbe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:17:19.220594 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189a1486-a2fe-47cc-8fbc-af7afda5adbe admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:17:19.231363 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 977/1958] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Aug 24 06:17:19 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 06:17:19.235663 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e52f3b5-eb4c-43cc-a993-8079becfcc3f admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:17:19.246906 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e52f3b5-eb4c-43cc-a993-8079becfcc3f admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:17:19.252035 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684c5f72-ff75-4609-9bd9-ac21cf486683 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:17:19.253022 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684c5f72-ff75-4609-9bd9-ac21cf486683 admin admin] REQUEST_METHOD: `PUT` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:19.253352 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684c5f72-ff75-4609-9bd9-ac21cf486683 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:19.253724 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684c5f72-ff75-4609-9bd9-ac21cf486683 admin admin] PATH_INFO: `/v3/projects/7d37379ce6c24acf83bd68e06fca20f8/users/d2b44c8575f54d6a82f40b9c3b70c38e/roles/49e47cd157d743f6905f1db07ec878a7` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:19.260745 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684c5f72-ff75-4609-9bd9-ac21cf486683 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d37379ce6c24acf83bd68e06fca20f8, user_id=d2b44c8575f54d6a82f40b9c3b70c38e, role_id=49e47cd157d743f6905f1db07ec878a7)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:17:19.260745 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684c5f72-ff75-4609-9bd9-ac21cf486683 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:17:19.277954 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 982/1959] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Aug 24 06:17:19 2022] PUT /identity/v3/projects/7d37379ce6c24acf83bd68e06fca20f8/users/d2b44c8575f54d6a82f40b9c3b70c38e/roles/49e47cd157d743f6905f1db07ec878a7 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:17:19.488989 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189a1486-a2fe-47cc-8fbc-af7afda5adbe admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:17:19.490765 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-189a1486-a2fe-47cc-8fbc-af7afda5adbe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:17:19.510066 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189a1486-a2fe-47cc-8fbc-af7afda5adbe admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:17:19.600658 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d5594f-d2ac-4128-9a55-38e8fec73799 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:17:19.603304 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d5594f-d2ac-4128-9a55-38e8fec73799 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:19.603735 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d5594f-d2ac-4128-9a55-38e8fec73799 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:19.604106 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d5594f-d2ac-4128-9a55-38e8fec73799 admin admin] PATH_INFO: `/v3/users/a9cea11d6d244d0da7129f658ac6655e` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:19.605762 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d5594f-d2ac-4128-9a55-38e8fec73799 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a9cea11d6d244d0da7129f658ac6655e)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:17:19.609050 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d5594f-d2ac-4128-9a55-38e8fec73799 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:17:19.802929 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:17:19.827572 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-b4d5594f-d2ac-4128-9a55-38e8fec73799 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a9cea11d6d244d0da7129f658ac6655e'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:17:19.839747 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-b4d5594f-d2ac-4128-9a55-38e8fec73799 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a9cea11d6d244d0da7129f658ac6655e'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:17:19.840622 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-b4d5594f-d2ac-4128-9a55-38e8fec73799 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a9cea11d6d244d0da7129f658ac6655e'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:17:19.919949 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-b4d5594f-d2ac-4128-9a55-38e8fec73799 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a9cea11d6d244d0da7129f658ac6655e'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:17:19.939779 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-b4d5594f-d2ac-4128-9a55-38e8fec73799 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a9cea11d6d244d0da7129f658ac6655e'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:17:19.973975 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 978/1960] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Aug 24 06:17:19 2022] DELETE /identity/v3/users/a9cea11d6d244d0da7129f658ac6655e => generated 0 bytes in 485 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:17:19.977659 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea69b52-3a6d-4dfa-8ca7-a27e4d106937 None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:19.978140 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea69b52-3a6d-4dfa-8ca7-a27e4d106937 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:19.978216 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea69b52-3a6d-4dfa-8ca7-a27e4d106937 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:20.004206 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-4ea69b52-3a6d-4dfa-8ca7-a27e4d106937 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:17:20.005535 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-4ea69b52-3a6d-4dfa-8ca7-a27e4d106937 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:17:20.085557 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 983/1961] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:17:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:17:20.095123 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d5594f-d2ac-4128-9a55-38e8fec73799 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:17:20.097305 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-b4d5594f-d2ac-4128-9a55-38e8fec73799 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:17:20.190617 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d5594f-d2ac-4128-9a55-38e8fec73799 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:17:20.195288 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f95d34-1742-4216-a4f1-61eeb6581762 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bb6c79db75e4d169331c02133d1ec32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:17:20.196007 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f95d34-1742-4216-a4f1-61eeb6581762 service neutron] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:20.196608 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f95d34-1742-4216-a4f1-61eeb6581762 service neutron] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:20.198753 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f95d34-1742-4216-a4f1-61eeb6581762 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:20.208262 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f95d34-1742-4216-a4f1-61eeb6581762 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:17:20.209760 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f95d34-1742-4216-a4f1-61eeb6581762 service neutron] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:17:20.228710 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 979/1962] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:17:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:17:20.275439 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:17:20.885354 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea69b52-3a6d-4dfa-8ca7-a27e4d106937 None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:17:20.887321 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-4ea69b52-3a6d-4dfa-8ca7-a27e4d106937 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:17:20.951457 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea69b52-3a6d-4dfa-8ca7-a27e4d106937 None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:17:20.975116 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b48508-8133-4e3c-8a4e-eafc9b63bce7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:17:20.976104 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b48508-8133-4e3c-8a4e-eafc9b63bce7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:20.981308 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b48508-8133-4e3c-8a4e-eafc9b63bce7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:20.981878 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b48508-8133-4e3c-8a4e-eafc9b63bce7 admin admin] PATH_INFO: `/v3/projects/183f907b77d84b4e89f32df021315633` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:20.990483 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b48508-8133-4e3c-8a4e-eafc9b63bce7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=183f907b77d84b4e89f32df021315633)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:17:20.994271 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b48508-8133-4e3c-8a4e-eafc9b63bce7 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:17:21.176473 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:17:21.212962 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-f1b48508-8133-4e3c-8a4e-eafc9b63bce7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '183f907b77d84b4e89f32df021315633', 'request_id': 'req-f1b48508-8133-4e3c-8a4e-eafc9b63bce7'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:17:21.225970 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-f1b48508-8133-4e3c-8a4e-eafc9b63bce7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '183f907b77d84b4e89f32df021315633', 'request_id': 'req-f1b48508-8133-4e3c-8a4e-eafc9b63bce7'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:17:21.235245 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-f1b48508-8133-4e3c-8a4e-eafc9b63bce7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '183f907b77d84b4e89f32df021315633', 'request_id': 'req-f1b48508-8133-4e3c-8a4e-eafc9b63bce7'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:17:21.309610 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-f1b48508-8133-4e3c-8a4e-eafc9b63bce7 admin admin] The token cache is being invalidate because project 183f907b77d84b4e89f32df021315633 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81851) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:17:21.310030 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-f1b48508-8133-4e3c-8a4e-eafc9b63bce7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:17:21.311364 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-f1b48508-8133-4e3c-8a4e-eafc9b63bce7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:17:21.317468 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 984/1963] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Aug 24 06:17:20 2022] DELETE /identity/v3/projects/183f907b77d84b4e89f32df021315633 => generated 0 bytes in 430 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:17:21.319186 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f95d34-1742-4216-a4f1-61eeb6581762 service neutron] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:17:21.330324 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f95d34-1742-4216-a4f1-61eeb6581762 service neutron] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:17:21.466964 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7a0092-3957-45cc-b237-2f078ae5ea66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:17:21.467846 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7a0092-3957-45cc-b237-2f078ae5ea66 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:21.468408 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7a0092-3957-45cc-b237-2f078ae5ea66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:21.468870 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7a0092-3957-45cc-b237-2f078ae5ea66 admin admin] PATH_INFO: `/v3/users/0f252d1d6e124be29a369dc557bbbfa3` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:21.470423 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7a0092-3957-45cc-b237-2f078ae5ea66 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0f252d1d6e124be29a369dc557bbbfa3)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:17:21.473348 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7a0092-3957-45cc-b237-2f078ae5ea66 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:17:21.698832 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-bf7a0092-3957-45cc-b237-2f078ae5ea66 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0f252d1d6e124be29a369dc557bbbfa3'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:17:21.716393 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-bf7a0092-3957-45cc-b237-2f078ae5ea66 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0f252d1d6e124be29a369dc557bbbfa3'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:17:21.717178 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-bf7a0092-3957-45cc-b237-2f078ae5ea66 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0f252d1d6e124be29a369dc557bbbfa3'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:17:21.803548 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-bf7a0092-3957-45cc-b237-2f078ae5ea66 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0f252d1d6e124be29a369dc557bbbfa3'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:17:21.818711 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-bf7a0092-3957-45cc-b237-2f078ae5ea66 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0f252d1d6e124be29a369dc557bbbfa3'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:17:21.828137 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 980/1964] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Aug 24 06:17:21 2022] DELETE /identity/v3/users/0f252d1d6e124be29a369dc557bbbfa3 => generated 0 bytes in 509 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:17:22.269442 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:17:22.617519 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b48508-8133-4e3c-8a4e-eafc9b63bce7 admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:17:22.619695 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-f1b48508-8133-4e3c-8a4e-eafc9b63bce7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:17:22.646254 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b48508-8133-4e3c-8a4e-eafc9b63bce7 admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:17:22.765369 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717cd49f-6978-4c3e-891a-5e75b8cc8d9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:17:22.766235 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717cd49f-6978-4c3e-891a-5e75b8cc8d9c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:22.766799 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717cd49f-6978-4c3e-891a-5e75b8cc8d9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:22.774378 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717cd49f-6978-4c3e-891a-5e75b8cc8d9c admin admin] PATH_INFO: `/v3/projects/d4d9ccdd0928458894a127cb4ef2af5c` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:22.774378 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717cd49f-6978-4c3e-891a-5e75b8cc8d9c admin admin] RBAC: Authorizing `identity:delete_project(project_id=d4d9ccdd0928458894a127cb4ef2af5c)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:17:22.774378 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717cd49f-6978-4c3e-891a-5e75b8cc8d9c admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:17:22.902895 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-717cd49f-6978-4c3e-891a-5e75b8cc8d9c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd4d9ccdd0928458894a127cb4ef2af5c', 'request_id': 'req-717cd49f-6978-4c3e-891a-5e75b8cc8d9c'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:17:22.909017 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-717cd49f-6978-4c3e-891a-5e75b8cc8d9c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd4d9ccdd0928458894a127cb4ef2af5c', 'request_id': 'req-717cd49f-6978-4c3e-891a-5e75b8cc8d9c'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:17:22.915600 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-717cd49f-6978-4c3e-891a-5e75b8cc8d9c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd4d9ccdd0928458894a127cb4ef2af5c', 'request_id': 'req-717cd49f-6978-4c3e-891a-5e75b8cc8d9c'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:17:23.012677 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-717cd49f-6978-4c3e-891a-5e75b8cc8d9c admin admin] The token cache is being invalidate because project d4d9ccdd0928458894a127cb4ef2af5c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81851) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:17:23.012963 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-717cd49f-6978-4c3e-891a-5e75b8cc8d9c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:17:23.013780 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-717cd49f-6978-4c3e-891a-5e75b8cc8d9c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:17:23.015545 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 985/1965] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Aug 24 06:17:22 2022] DELETE /identity/v3/projects/d4d9ccdd0928458894a127cb4ef2af5c => generated 0 bytes in 399 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:17:23.020608 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1824029d-7162-4bdb-ad6b-6b4cec5e3709 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:23.021146 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1824029d-7162-4bdb-ad6b-6b4cec5e3709 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:23.021146 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1824029d-7162-4bdb-ad6b-6b4cec5e3709 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:23.041081 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-1824029d-7162-4bdb-ad6b-6b4cec5e3709 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:17:23.042940 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-1824029d-7162-4bdb-ad6b-6b4cec5e3709 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:17:23.107704 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 981/1966] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:17:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 06:17:23.117611 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4c0f0d-9448-46a1-ab2d-45db3ead5baa None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:23.118006 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4c0f0d-9448-46a1-ab2d-45db3ead5baa None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:23.118275 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4c0f0d-9448-46a1-ab2d-45db3ead5baa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:23.138692 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:17:23.151976 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-ce4c0f0d-9448-46a1-ab2d-45db3ead5baa None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:17:23.153373 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-ce4c0f0d-9448-46a1-ab2d-45db3ead5baa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:17:23.164768 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 986/1967] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:17:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 06:17:23.171229 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1824029d-7162-4bdb-ad6b-6b4cec5e3709 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:17:23.184201 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1824029d-7162-4bdb-ad6b-6b4cec5e3709 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:17:23.194791 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d75e0b-d383-4a81-b9c0-e421b06753dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:17:23.194791 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d75e0b-d383-4a81-b9c0-e421b06753dd admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:23.194791 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d75e0b-d383-4a81-b9c0-e421b06753dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:23.194791 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d75e0b-d383-4a81-b9c0-e421b06753dd admin admin] PATH_INFO: `/v3/domains` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:23.195198 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d75e0b-d383-4a81-b9c0-e421b06753dd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:17:23.195198 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d75e0b-d383-4a81-b9c0-e421b06753dd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:17:23.195198 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d75e0b-d383-4a81-b9c0-e421b06753dd admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:17:23.208628 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 982/1968] 10.222.0.62 () {58 vars in 1153 bytes} [Wed Aug 24 06:17:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:17:23.213558 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4c0f0d-9448-46a1-ab2d-45db3ead5baa None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:17:23.236805 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4c0f0d-9448-46a1-ab2d-45db3ead5baa None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:17:23.244156 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df1715e-178b-46d3-b330-a132c3b28dfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:17:23.245720 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df1715e-178b-46d3-b330-a132c3b28dfb admin admin] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:23.247675 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df1715e-178b-46d3-b330-a132c3b28dfb admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:23.247675 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df1715e-178b-46d3-b330-a132c3b28dfb admin admin] PATH_INFO: `/v3/projects` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:23.247675 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df1715e-178b-46d3-b330-a132c3b28dfb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:17:23.262379 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df1715e-178b-46d3-b330-a132c3b28dfb admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:17:23.376063 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 987/1969] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Aug 24 06:17:23 2022] POST /identity/v3/projects => generated 401 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:17:23.379933 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d75e0b-d383-4a81-b9c0-e421b06753dd admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:17:23.399231 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d75e0b-d383-4a81-b9c0-e421b06753dd admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:17:23.497723 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37d2fb3-5032-4f9c-b30c-b3b13ca09938 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:17:23.498408 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37d2fb3-5032-4f9c-b30c-b3b13ca09938 admin admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:23.498849 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37d2fb3-5032-4f9c-b30c-b3b13ca09938 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:23.499130 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37d2fb3-5032-4f9c-b30c-b3b13ca09938 admin admin] PATH_INFO: `/v3/users` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:23.499965 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37d2fb3-5032-4f9c-b30c-b3b13ca09938 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:17:23.501982 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37d2fb3-5032-4f9c-b30c-b3b13ca09938 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:17:23.632533 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 983/1970] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Aug 24 06:17:23 2022] POST /identity/v3/users => generated 301 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:17:23.637360 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df1715e-178b-46d3-b330-a132c3b28dfb admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:17:23.655545 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df1715e-178b-46d3-b330-a132c3b28dfb admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:17:23.664562 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee3a3f4-b440-44c3-a981-f21dee256071 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:17:23.665249 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee3a3f4-b440-44c3-a981-f21dee256071 admin admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:23.665629 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee3a3f4-b440-44c3-a981-f21dee256071 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:23.666000 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee3a3f4-b440-44c3-a981-f21dee256071 admin admin] PATH_INFO: `/v3/roles` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:23.667089 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee3a3f4-b440-44c3-a981-f21dee256071 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:17:23.668985 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee3a3f4-b440-44c3-a981-f21dee256071 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:17:23.686312 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 988/1971] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Aug 24 06:17:23 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:17:23.691668 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37d2fb3-5032-4f9c-b30c-b3b13ca09938 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:17:23.711636 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37d2fb3-5032-4f9c-b30c-b3b13ca09938 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:17:23.717462 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4962fd8-1629-48fb-928a-6f6d3e591082 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:17:23.718194 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4962fd8-1629-48fb-928a-6f6d3e591082 admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:23.718575 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4962fd8-1629-48fb-928a-6f6d3e591082 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:23.718951 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4962fd8-1629-48fb-928a-6f6d3e591082 admin admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:23.719702 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4962fd8-1629-48fb-928a-6f6d3e591082 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:17:23.722278 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4962fd8-1629-48fb-928a-6f6d3e591082 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:17:23.739784 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 984/1972] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Aug 24 06:17:23 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:17:23.742089 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee3a3f4-b440-44c3-a981-f21dee256071 admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:17:23.757891 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee3a3f4-b440-44c3-a981-f21dee256071 admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:17:23.765707 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435d9ab3-da57-4798-bc0f-39a4d8848f91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:17:23.770404 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435d9ab3-da57-4798-bc0f-39a4d8848f91 admin admin] REQUEST_METHOD: `PUT` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:23.770404 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435d9ab3-da57-4798-bc0f-39a4d8848f91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:23.770404 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435d9ab3-da57-4798-bc0f-39a4d8848f91 admin admin] PATH_INFO: `/v3/projects/21a786448e1c40d98216c6d6e2c9ada0/users/fc432a6e990541a9a656a1c70c0183a5/roles/580aa7dbb38d4cff8f9d1bb8983f99ef` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:23.786212 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435d9ab3-da57-4798-bc0f-39a4d8848f91 admin admin] RBAC: Authorizing `identity:create_grant(project_id=21a786448e1c40d98216c6d6e2c9ada0, user_id=fc432a6e990541a9a656a1c70c0183a5, role_id=580aa7dbb38d4cff8f9d1bb8983f99ef)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:17:23.790181 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435d9ab3-da57-4798-bc0f-39a4d8848f91 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:17:23.819046 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:17:23.895189 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 989/1973] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Aug 24 06:17:23 2022] PUT /identity/v3/projects/21a786448e1c40d98216c6d6e2c9ada0/users/fc432a6e990541a9a656a1c70c0183a5/roles/580aa7dbb38d4cff8f9d1bb8983f99ef => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:17:23.906446 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998959d5-a043-4493-97f0-a5cfeabf89d4 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:23.906758 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998959d5-a043-4493-97f0-a5cfeabf89d4 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:23.907078 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998959d5-a043-4493-97f0-a5cfeabf89d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:23.939205 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-998959d5-a043-4493-97f0-a5cfeabf89d4 None None] MFA Rules not processed for user `fc432a6e990541a9a656a1c70c0183a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:17:23.944878 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-998959d5-a043-4493-97f0-a5cfeabf89d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:17:24.028209 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 985/1974] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:17:23 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:17:24.035581 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435d9ab3-da57-4798-bc0f-39a4d8848f91 admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:17:24.130830 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435d9ab3-da57-4798-bc0f-39a4d8848f91 admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:17:24.136477 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc99a01-5484-403c-a249-fdf190034458 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:17:24.137142 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc99a01-5484-403c-a249-fdf190034458 admin admin] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:24.137426 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc99a01-5484-403c-a249-fdf190034458 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:24.137703 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc99a01-5484-403c-a249-fdf190034458 admin admin] PATH_INFO: `/v3/projects` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:24.138382 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc99a01-5484-403c-a249-fdf190034458 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:17:24.140902 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc99a01-5484-403c-a249-fdf190034458 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:17:24.240603 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 990/1975] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Aug 24 06:17:24 2022] POST /identity/v3/projects => generated 399 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:17:24.242761 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998959d5-a043-4493-97f0-a5cfeabf89d4 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:17:24.254289 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998959d5-a043-4493-97f0-a5cfeabf89d4 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:17:24.313296 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:17:24.326602 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a768df3-4901-4f4d-8274-537abb0a1ec5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:17:24.327463 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a768df3-4901-4f4d-8274-537abb0a1ec5 admin admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:24.327972 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a768df3-4901-4f4d-8274-537abb0a1ec5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:24.328624 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a768df3-4901-4f4d-8274-537abb0a1ec5 admin admin] PATH_INFO: `/v3/users` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:24.329530 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a768df3-4901-4f4d-8274-537abb0a1ec5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:17:24.331914 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a768df3-4901-4f4d-8274-537abb0a1ec5 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:17:24.474436 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 986/1976] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Aug 24 06:17:24 2022] POST /identity/v3/users => generated 306 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 06:17:24.475285 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc99a01-5484-403c-a249-fdf190034458 admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:17:24.486470 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc99a01-5484-403c-a249-fdf190034458 admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:17:24.492223 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b6d8b6-c13e-4feb-a02f-740a5edb9a79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:17:24.493058 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b6d8b6-c13e-4feb-a02f-740a5edb9a79 admin admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:24.493442 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b6d8b6-c13e-4feb-a02f-740a5edb9a79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:24.493824 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b6d8b6-c13e-4feb-a02f-740a5edb9a79 admin admin] PATH_INFO: `/v3/roles` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:24.494580 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b6d8b6-c13e-4feb-a02f-740a5edb9a79 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:17:24.496488 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b6d8b6-c13e-4feb-a02f-740a5edb9a79 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:17:24.509143 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 991/1977] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Aug 24 06:17:24 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:17:24.515708 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a768df3-4901-4f4d-8274-537abb0a1ec5 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:17:24.568102 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a768df3-4901-4f4d-8274-537abb0a1ec5 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:17:24.611496 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0dc78d-1b3c-4f72-b3d7-5f29ec16e0d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:17:24.613545 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0dc78d-1b3c-4f72-b3d7-5f29ec16e0d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:24.614013 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0dc78d-1b3c-4f72-b3d7-5f29ec16e0d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:24.614497 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0dc78d-1b3c-4f72-b3d7-5f29ec16e0d9 admin admin] PATH_INFO: `/v3/projects/d914df5358d64502be8cd74a8a653003/users/ee36e0ddf7a141b6a03bed0dc09ab3c9/roles/49e47cd157d743f6905f1db07ec878a7` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:24.683973 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0dc78d-1b3c-4f72-b3d7-5f29ec16e0d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d914df5358d64502be8cd74a8a653003, user_id=ee36e0ddf7a141b6a03bed0dc09ab3c9, role_id=49e47cd157d743f6905f1db07ec878a7)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:17:24.694732 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0dc78d-1b3c-4f72-b3d7-5f29ec16e0d9 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:17:24.812330 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b6d8b6-c13e-4feb-a02f-740a5edb9a79 admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:17:24.813157 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 987/1978] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Aug 24 06:17:24 2022] PUT /identity/v3/projects/d914df5358d64502be8cd74a8a653003/users/ee36e0ddf7a141b6a03bed0dc09ab3c9/roles/49e47cd157d743f6905f1db07ec878a7 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:17:24.824682 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b6d8b6-c13e-4feb-a02f-740a5edb9a79 admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:17:24.870546 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0dc78d-1b3c-4f72-b3d7-5f29ec16e0d9 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:17:24.875683 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-cd0dc78d-1b3c-4f72-b3d7-5f29ec16e0d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:17:24.898379 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:17:24.944452 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7423e66-8277-469c-b952-bbe2b81ec043 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:17:24.945288 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7423e66-8277-469c-b952-bbe2b81ec043 admin admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:24.945643 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7423e66-8277-469c-b952-bbe2b81ec043 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:24.946632 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7423e66-8277-469c-b952-bbe2b81ec043 admin admin] PATH_INFO: `/v3/roles` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:24.946632 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7423e66-8277-469c-b952-bbe2b81ec043 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:17:24.948881 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7423e66-8277-469c-b952-bbe2b81ec043 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:17:24.966639 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 992/1979] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Aug 24 06:17:24 2022] GET /identity/v3/roles => generated 1420 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:17:24.967762 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7423e66-8277-469c-b952-bbe2b81ec043 admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:17:24.979549 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7423e66-8277-469c-b952-bbe2b81ec043 admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:17:24.985171 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057d8628-caf1-418d-9726-b2654b6d9f82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:17:24.986021 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057d8628-caf1-418d-9726-b2654b6d9f82 admin admin] REQUEST_METHOD: `PUT` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:24.986376 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057d8628-caf1-418d-9726-b2654b6d9f82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:24.987302 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057d8628-caf1-418d-9726-b2654b6d9f82 admin admin] PATH_INFO: `/v3/projects/d914df5358d64502be8cd74a8a653003/users/ee36e0ddf7a141b6a03bed0dc09ab3c9/roles/49e47cd157d743f6905f1db07ec878a7` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:24.987697 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0dc78d-1b3c-4f72-b3d7-5f29ec16e0d9 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:17:24.989516 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057d8628-caf1-418d-9726-b2654b6d9f82 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d914df5358d64502be8cd74a8a653003, user_id=ee36e0ddf7a141b6a03bed0dc09ab3c9, role_id=49e47cd157d743f6905f1db07ec878a7)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:17:24.992812 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fd2dc0-0d7b-4c12-8879-babca7287828 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c4def55ac0e48b29836bc5c64bc7402', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:17:24.993021 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057d8628-caf1-418d-9726-b2654b6d9f82 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:17:24.993193 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fd2dc0-0d7b-4c12-8879-babca7287828 service cinder] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:24.993193 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fd2dc0-0d7b-4c12-8879-babca7287828 service cinder] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:24.993438 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fd2dc0-0d7b-4c12-8879-babca7287828 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:24.995905 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-07fd2dc0-0d7b-4c12-8879-babca7287828 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:17:25.029821 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 993/1980] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Aug 24 06:17:24 2022] PUT /identity/v3/projects/d914df5358d64502be8cd74a8a653003/users/ee36e0ddf7a141b6a03bed0dc09ab3c9/roles/49e47cd157d743f6905f1db07ec878a7 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:17:25.034696 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7078395c-d736-4b55-bdf7-6ad8bf5f5b29 None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:25.036239 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7078395c-d736-4b55-bdf7-6ad8bf5f5b29 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:25.036705 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7078395c-d736-4b55-bdf7-6ad8bf5f5b29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:25.080653 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fd2dc0-0d7b-4c12-8879-babca7287828 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:17:25.083703 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fd2dc0-0d7b-4c12-8879-babca7287828 service cinder] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:17:25.105939 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-7078395c-d736-4b55-bdf7-6ad8bf5f5b29 None None] MFA Rules not processed for user `ee36e0ddf7a141b6a03bed0dc09ab3c9`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:17:25.108873 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 988/1981] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:17:24 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 240 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:17:25.109584 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-7078395c-d736-4b55-bdf7-6ad8bf5f5b29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:17:25.282038 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 994/1982] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:17:25 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:17:25.337408 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fd2dc0-0d7b-4c12-8879-babca7287828 service cinder] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:17:25.347709 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fd2dc0-0d7b-4c12-8879-babca7287828 service cinder] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:17:25.560002 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9a74df-6302-41eb-b19c-d114779bca87 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c4def55ac0e48b29836bc5c64bc7402', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:17:25.560900 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9a74df-6302-41eb-b19c-d114779bca87 service cinder] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:25.561398 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9a74df-6302-41eb-b19c-d114779bca87 service cinder] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:25.561879 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9a74df-6302-41eb-b19c-d114779bca87 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:25.587719 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9a74df-6302-41eb-b19c-d114779bca87 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:17:25.589809 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9a74df-6302-41eb-b19c-d114779bca87 service cinder] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:17:25.625176 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 989/1983] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:17:25 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 06:17:25.969448 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7078395c-d736-4b55-bdf7-6ad8bf5f5b29 None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:17:25.984869 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7078395c-d736-4b55-bdf7-6ad8bf5f5b29 None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:17:25.994095 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4b8724-6262-4d5c-bf31-3dafe9ab495d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c4def55ac0e48b29836bc5c64bc7402', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:17:25.994749 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4b8724-6262-4d5c-bf31-3dafe9ab495d service cinder] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:25.995129 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4b8724-6262-4d5c-bf31-3dafe9ab495d service cinder] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:25.995407 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4b8724-6262-4d5c-bf31-3dafe9ab495d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:26.060792 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:17:26.116326 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4b8724-6262-4d5c-bf31-3dafe9ab495d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:17:26.118291 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4b8724-6262-4d5c-bf31-3dafe9ab495d service cinder] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:17:26.129817 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 995/1984] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:17:25 2022] GET /identity/v3/auth/tokens => generated 3278 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 06:17:26.308262 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:17:27.824767 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:17:28.071929 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9a74df-6302-41eb-b19c-d114779bca87 service cinder] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:17:28.073791 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-0c9a74df-6302-41eb-b19c-d114779bca87 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:17:28.184536 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778c792d-41d4-4d1f-a164-63022714d52c None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:28.185306 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778c792d-41d4-4d1f-a164-63022714d52c None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:28.186248 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778c792d-41d4-4d1f-a164-63022714d52c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:28.200538 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9a74df-6302-41eb-b19c-d114779bca87 service cinder] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:17:28.205536 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c2895f-fb53-4757-a522-52e2b1219e67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:17:28.208165 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c2895f-fb53-4757-a522-52e2b1219e67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:28.208165 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c2895f-fb53-4757-a522-52e2b1219e67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:28.208165 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c2895f-fb53-4757-a522-52e2b1219e67 admin admin] PATH_INFO: `/v3/users/0e56d56a9a80412a839f5376d7c116f7` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:28.208165 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c2895f-fb53-4757-a522-52e2b1219e67 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0e56d56a9a80412a839f5376d7c116f7)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:17:28.210002 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c2895f-fb53-4757-a522-52e2b1219e67 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:17:28.231973 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-778c792d-41d4-4d1f-a164-63022714d52c None None] MFA Rules not processed for user `d2b44c8575f54d6a82f40b9c3b70c38e`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:17:28.233487 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-778c792d-41d4-4d1f-a164-63022714d52c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:17:28.325785 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 996/1985] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:17:28 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:17:28.338479 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778c792d-41d4-4d1f-a164-63022714d52c None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:17:28.341225 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-778c792d-41d4-4d1f-a164-63022714d52c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:17:28.460552 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778c792d-41d4-4d1f-a164-63022714d52c None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:17:28.464669 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74252ee-8652-4cd0-bd63-e6768cb60e03 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7d52162cbdc4218b577ecc1e69a88c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:17:28.465302 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74252ee-8652-4cd0-bd63-e6768cb60e03 service glance] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:28.465591 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74252ee-8652-4cd0-bd63-e6768cb60e03 service glance] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:28.465869 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74252ee-8652-4cd0-bd63-e6768cb60e03 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:28.467920 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-a74252ee-8652-4cd0-bd63-e6768cb60e03 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:17:28.472761 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-56c2895f-fb53-4757-a522-52e2b1219e67 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0e56d56a9a80412a839f5376d7c116f7'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:17:28.485024 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-56c2895f-fb53-4757-a522-52e2b1219e67 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0e56d56a9a80412a839f5376d7c116f7'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:17:28.485824 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-56c2895f-fb53-4757-a522-52e2b1219e67 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0e56d56a9a80412a839f5376d7c116f7'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:17:28.547095 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74252ee-8652-4cd0-bd63-e6768cb60e03 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:17:28.548697 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74252ee-8652-4cd0-bd63-e6768cb60e03 service glance] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:17:28.560952 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 997/1986] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:17:28 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:17:28.635573 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-56c2895f-fb53-4757-a522-52e2b1219e67 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0e56d56a9a80412a839f5376d7c116f7'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:17:28.648155 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:17:28.661827 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-56c2895f-fb53-4757-a522-52e2b1219e67 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0e56d56a9a80412a839f5376d7c116f7'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:17:28.670392 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 990/1987] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Aug 24 06:17:28 2022] DELETE /identity/v3/users/0e56d56a9a80412a839f5376d7c116f7 => generated 0 bytes in 600 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:17:28.675913 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817f797a-3ab4-4068-86cc-53b2ad118917 None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:28.676171 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817f797a-3ab4-4068-86cc-53b2ad118917 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:28.676442 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817f797a-3ab4-4068-86cc-53b2ad118917 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:28.711000 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-817f797a-3ab4-4068-86cc-53b2ad118917 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:17:28.713418 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-817f797a-3ab4-4068-86cc-53b2ad118917 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:17:28.809958 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 998/1988] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:17:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:17:28.823948 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c2895f-fb53-4757-a522-52e2b1219e67 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:17:28.826806 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-56c2895f-fb53-4757-a522-52e2b1219e67 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:17:28.902477 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c2895f-fb53-4757-a522-52e2b1219e67 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:17:28.909887 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f69b95d-2264-4f94-957f-ec8c9800543b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bb6c79db75e4d169331c02133d1ec32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:17:28.910706 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f69b95d-2264-4f94-957f-ec8c9800543b service neutron] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:28.911045 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f69b95d-2264-4f94-957f-ec8c9800543b service neutron] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:28.911701 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f69b95d-2264-4f94-957f-ec8c9800543b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:28.916235 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aceafdeb-5082-42ae-920b-94f550aefba6 None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:28.917117 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aceafdeb-5082-42ae-920b-94f550aefba6 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:28.917117 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aceafdeb-5082-42ae-920b-94f550aefba6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:28.929468 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f69b95d-2264-4f94-957f-ec8c9800543b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:17:28.931397 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f69b95d-2264-4f94-957f-ec8c9800543b service neutron] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:17:28.938584 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-aceafdeb-5082-42ae-920b-94f550aefba6 None None] MFA Rules not processed for user `b82726a12436435cb931b986ef884c2e`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:17:28.940281 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-aceafdeb-5082-42ae-920b-94f550aefba6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:17:28.952198 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 991/1989] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:17:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:17:28.962802 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:17:29.014813 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 999/1990] 10.222.0.62 () {58 vars in 1009 bytes} [Wed Aug 24 06:17:28 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 06:17:29.032751 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f69b95d-2264-4f94-957f-ec8c9800543b service neutron] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:17:29.035171 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-3f69b95d-2264-4f94-957f-ec8c9800543b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:17:29.141009 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f69b95d-2264-4f94-957f-ec8c9800543b service neutron] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:17:29.152003 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f41d7b-8b6b-4be1-a51a-ff83c1f18b97 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14d4fed04fb144d08918bd5d4f808d23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:17:29.152962 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f41d7b-8b6b-4be1-a51a-ff83c1f18b97 service swift] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:29.153349 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f41d7b-8b6b-4be1-a51a-ff83c1f18b97 service swift] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:29.153724 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f41d7b-8b6b-4be1-a51a-ff83c1f18b97 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:29.163088 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f41d7b-8b6b-4be1-a51a-ff83c1f18b97 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:17:29.164839 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f41d7b-8b6b-4be1-a51a-ff83c1f18b97 service swift] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:17:29.174674 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 992/1991] 10.222.0.62 () {60 vars in 1384 bytes} [Wed Aug 24 06:17:29 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 06:17:29.601323 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:17:29.659458 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:17:29.980666 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aceafdeb-5082-42ae-920b-94f550aefba6 None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:17:29.983509 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-aceafdeb-5082-42ae-920b-94f550aefba6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:17:30.039221 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aceafdeb-5082-42ae-920b-94f550aefba6 None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:17:30.057943 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8d2496-bbfe-4d51-80fa-2a0dbab92a30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:17:30.060828 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8d2496-bbfe-4d51-80fa-2a0dbab92a30 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:30.063474 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8d2496-bbfe-4d51-80fa-2a0dbab92a30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:30.064124 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8d2496-bbfe-4d51-80fa-2a0dbab92a30 admin admin] PATH_INFO: `/v3/projects/69ab491edfef411eaef0bea127a0ae73` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:30.068870 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8d2496-bbfe-4d51-80fa-2a0dbab92a30 admin admin] RBAC: Authorizing `identity:delete_project(project_id=69ab491edfef411eaef0bea127a0ae73)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:17:30.076380 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8d2496-bbfe-4d51-80fa-2a0dbab92a30 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:17:30.418002 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-7a8d2496-bbfe-4d51-80fa-2a0dbab92a30 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '69ab491edfef411eaef0bea127a0ae73', 'request_id': 'req-7a8d2496-bbfe-4d51-80fa-2a0dbab92a30'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:17:30.437546 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-7a8d2496-bbfe-4d51-80fa-2a0dbab92a30 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '69ab491edfef411eaef0bea127a0ae73', 'request_id': 'req-7a8d2496-bbfe-4d51-80fa-2a0dbab92a30'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:17:30.461643 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-7a8d2496-bbfe-4d51-80fa-2a0dbab92a30 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '69ab491edfef411eaef0bea127a0ae73', 'request_id': 'req-7a8d2496-bbfe-4d51-80fa-2a0dbab92a30'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:17:30.826358 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-7a8d2496-bbfe-4d51-80fa-2a0dbab92a30 admin admin] The token cache is being invalidate because project 69ab491edfef411eaef0bea127a0ae73 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81851) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:17:30.829029 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-7a8d2496-bbfe-4d51-80fa-2a0dbab92a30 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:17:30.831572 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-7a8d2496-bbfe-4d51-80fa-2a0dbab92a30 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:17:30.835072 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1000/1992] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Aug 24 06:17:29 2022] DELETE /identity/v3/projects/69ab491edfef411eaef0bea127a0ae73 => generated 0 bytes in 855 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:17:30.839316 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f41d7b-8b6b-4be1-a51a-ff83c1f18b97 service swift] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:17:30.858131 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f41d7b-8b6b-4be1-a51a-ff83c1f18b97 service swift] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:17:31.025255 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bd491c-0ad9-48fa-9f72-8f44985decbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:17:31.025914 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bd491c-0ad9-48fa-9f72-8f44985decbd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:31.026198 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bd491c-0ad9-48fa-9f72-8f44985decbd admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:31.026477 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bd491c-0ad9-48fa-9f72-8f44985decbd admin admin] PATH_INFO: `/v3/users/a9eb694acfd842369e2c398460187002` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:31.027822 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51bd491c-0ad9-48fa-9f72-8f44985decbd admin admin] RBAC: Authorizing `identity:delete_user(user_id=a9eb694acfd842369e2c398460187002)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:17:31.031326 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51bd491c-0ad9-48fa-9f72-8f44985decbd admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:17:31.069443 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:17:31.332484 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-51bd491c-0ad9-48fa-9f72-8f44985decbd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a9eb694acfd842369e2c398460187002'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:17:31.352722 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-51bd491c-0ad9-48fa-9f72-8f44985decbd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a9eb694acfd842369e2c398460187002'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:17:31.355802 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-51bd491c-0ad9-48fa-9f72-8f44985decbd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a9eb694acfd842369e2c398460187002'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:17:31.374427 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:17:31.498133 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-51bd491c-0ad9-48fa-9f72-8f44985decbd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a9eb694acfd842369e2c398460187002'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:17:31.524352 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-51bd491c-0ad9-48fa-9f72-8f44985decbd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a9eb694acfd842369e2c398460187002'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:17:31.541518 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 993/1993] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Aug 24 06:17:30 2022] DELETE /identity/v3/users/a9eb694acfd842369e2c398460187002 => generated 0 bytes in 698 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:17:31.625642 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=87,keystone:INSERT=6,keystone:DELETE=7 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:17:32.032795 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:17:32.281227 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:17:32.439431 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:17:32.716101 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8d2496-bbfe-4d51-80fa-2a0dbab92a30 admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:17:32.718892 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-7a8d2496-bbfe-4d51-80fa-2a0dbab92a30 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:17:32.763965 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8d2496-bbfe-4d51-80fa-2a0dbab92a30 admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:17:32.950937 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5697c41-29fa-4501-bd24-be18e9f4926c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:17:32.955104 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5697c41-29fa-4501-bd24-be18e9f4926c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:32.961540 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5697c41-29fa-4501-bd24-be18e9f4926c admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:32.961540 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5697c41-29fa-4501-bd24-be18e9f4926c admin admin] PATH_INFO: `/v3/projects/9f57110cd4574ca7b3eeecf20e1f0d1b` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:32.964186 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5697c41-29fa-4501-bd24-be18e9f4926c admin admin] RBAC: Authorizing `identity:delete_project(project_id=9f57110cd4574ca7b3eeecf20e1f0d1b)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:17:32.966396 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5697c41-29fa-4501-bd24-be18e9f4926c admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:17:33.165637 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:17:33.211057 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-e5697c41-29fa-4501-bd24-be18e9f4926c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9f57110cd4574ca7b3eeecf20e1f0d1b', 'request_id': 'req-e5697c41-29fa-4501-bd24-be18e9f4926c'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:17:33.224341 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-e5697c41-29fa-4501-bd24-be18e9f4926c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9f57110cd4574ca7b3eeecf20e1f0d1b', 'request_id': 'req-e5697c41-29fa-4501-bd24-be18e9f4926c'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:17:33.243327 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-e5697c41-29fa-4501-bd24-be18e9f4926c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9f57110cd4574ca7b3eeecf20e1f0d1b', 'request_id': 'req-e5697c41-29fa-4501-bd24-be18e9f4926c'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:17:33.346102 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-e5697c41-29fa-4501-bd24-be18e9f4926c admin admin] The token cache is being invalidate because project 9f57110cd4574ca7b3eeecf20e1f0d1b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81851) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:17:33.346102 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-e5697c41-29fa-4501-bd24-be18e9f4926c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:17:33.346102 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-e5697c41-29fa-4501-bd24-be18e9f4926c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:17:33.355019 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1001/1994] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Aug 24 06:17:32 2022] DELETE /identity/v3/projects/9f57110cd4574ca7b3eeecf20e1f0d1b => generated 0 bytes in 636 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:17:33.362004 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de1ed26-aa4c-4f91-8f56-e1c820638033 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:33.362943 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de1ed26-aa4c-4f91-8f56-e1c820638033 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:33.363848 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de1ed26-aa4c-4f91-8f56-e1c820638033 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:33.481390 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:17:33.497804 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-6de1ed26-aa4c-4f91-8f56-e1c820638033 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:17:33.500050 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-6de1ed26-aa4c-4f91-8f56-e1c820638033 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:17:33.805620 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 994/1995] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:17:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 450 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:17:33.808046 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050436b5-8082-49c8-abe5-aa3d959939b1 None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:33.808474 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050436b5-8082-49c8-abe5-aa3d959939b1 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:33.809017 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050436b5-8082-49c8-abe5-aa3d959939b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:33.874754 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-050436b5-8082-49c8-abe5-aa3d959939b1 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:17:33.876138 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-050436b5-8082-49c8-abe5-aa3d959939b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:17:33.902402 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1002/1996] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:17:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:17:33.903678 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de1ed26-aa4c-4f91-8f56-e1c820638033 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:17:33.926923 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de1ed26-aa4c-4f91-8f56-e1c820638033 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:17:33.940689 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d81bd60-a98a-4e4e-9e7b-bfa12f4e1775 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:17:33.941904 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d81bd60-a98a-4e4e-9e7b-bfa12f4e1775 admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:33.944883 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d81bd60-a98a-4e4e-9e7b-bfa12f4e1775 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:33.944883 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d81bd60-a98a-4e4e-9e7b-bfa12f4e1775 admin admin] PATH_INFO: `/v3/domains` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:33.944883 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d81bd60-a98a-4e4e-9e7b-bfa12f4e1775 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:17:33.950610 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d81bd60-a98a-4e4e-9e7b-bfa12f4e1775 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:17:33.953741 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d81bd60-a98a-4e4e-9e7b-bfa12f4e1775 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:17:33.989726 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 995/1997] 10.222.0.62 () {58 vars in 1153 bytes} [Wed Aug 24 06:17:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 06:17:33.994660 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050436b5-8082-49c8-abe5-aa3d959939b1 None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:17:34.010610 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050436b5-8082-49c8-abe5-aa3d959939b1 None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:17:34.022725 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f937aa-f90f-4149-9bf8-85adb2bd79b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:17:34.023856 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f937aa-f90f-4149-9bf8-85adb2bd79b8 admin admin] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:34.024372 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f937aa-f90f-4149-9bf8-85adb2bd79b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:34.029427 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f937aa-f90f-4149-9bf8-85adb2bd79b8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:34.030969 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f937aa-f90f-4149-9bf8-85adb2bd79b8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:17:34.038528 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f937aa-f90f-4149-9bf8-85adb2bd79b8 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:17:34.073323 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:17:34.258097 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1003/1998] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Aug 24 06:17:33 2022] POST /identity/v3/projects => generated 383 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:17:34.259158 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d81bd60-a98a-4e4e-9e7b-bfa12f4e1775 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:17:34.289740 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d81bd60-a98a-4e4e-9e7b-bfa12f4e1775 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:17:34.383951 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:17:34.458750 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:17:34.624009 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1440f40-c0de-4ae3-8e87-662d3377b26b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:17:34.625003 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1440f40-c0de-4ae3-8e87-662d3377b26b admin admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:34.625448 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1440f40-c0de-4ae3-8e87-662d3377b26b admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:34.627640 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1440f40-c0de-4ae3-8e87-662d3377b26b admin admin] PATH_INFO: `/v3/users` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:34.628678 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1440f40-c0de-4ae3-8e87-662d3377b26b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:17:34.632076 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1440f40-c0de-4ae3-8e87-662d3377b26b admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:17:34.774374 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 996/1999] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Aug 24 06:17:34 2022] POST /identity/v3/users => generated 292 bytes in 518 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:17:34.778525 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f937aa-f90f-4149-9bf8-85adb2bd79b8 admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:17:34.795686 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f937aa-f90f-4149-9bf8-85adb2bd79b8 admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:17:34.812096 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fedce07-9dee-4408-9663-9b7b801ea586 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:17:34.815446 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fedce07-9dee-4408-9663-9b7b801ea586 admin admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:34.816030 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fedce07-9dee-4408-9663-9b7b801ea586 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:34.818405 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fedce07-9dee-4408-9663-9b7b801ea586 admin admin] PATH_INFO: `/v3/roles` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:34.819360 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fedce07-9dee-4408-9663-9b7b801ea586 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:17:34.823396 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fedce07-9dee-4408-9663-9b7b801ea586 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:17:34.873130 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1440f40-c0de-4ae3-8e87-662d3377b26b admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:17:34.877329 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1004/2000] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Aug 24 06:17:34 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:17:34.888985 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1440f40-c0de-4ae3-8e87-662d3377b26b admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:17:34.913771 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786c67f1-9824-4cff-abe5-d9aefaa0060d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:17:34.915509 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786c67f1-9824-4cff-abe5-d9aefaa0060d admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:34.915797 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786c67f1-9824-4cff-abe5-d9aefaa0060d admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:34.916074 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786c67f1-9824-4cff-abe5-d9aefaa0060d admin admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:34.916772 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786c67f1-9824-4cff-abe5-d9aefaa0060d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:17:34.919138 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786c67f1-9824-4cff-abe5-d9aefaa0060d admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:17:34.964275 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 997/2001] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Aug 24 06:17:34 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:17:34.968938 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fedce07-9dee-4408-9663-9b7b801ea586 admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:17:34.983345 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fedce07-9dee-4408-9663-9b7b801ea586 admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:17:35.001628 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:17:35.008260 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170bdbd9-3525-46d0-97a7-29595330aeb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:17:35.008260 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170bdbd9-3525-46d0-97a7-29595330aeb9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:35.008260 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170bdbd9-3525-46d0-97a7-29595330aeb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:35.008260 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170bdbd9-3525-46d0-97a7-29595330aeb9 admin admin] PATH_INFO: `/v3/projects/1239f0f5e26d4b0d81f4f57df1b4d578/users/c3b90a10ca674c59a20171b30843a520/roles/580aa7dbb38d4cff8f9d1bb8983f99ef` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:35.052745 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170bdbd9-3525-46d0-97a7-29595330aeb9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1239f0f5e26d4b0d81f4f57df1b4d578, user_id=c3b90a10ca674c59a20171b30843a520, role_id=580aa7dbb38d4cff8f9d1bb8983f99ef)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:17:35.055902 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170bdbd9-3525-46d0-97a7-29595330aeb9 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:17:35.189122 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1005/2002] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Aug 24 06:17:34 2022] PUT /identity/v3/projects/1239f0f5e26d4b0d81f4f57df1b4d578/users/c3b90a10ca674c59a20171b30843a520/roles/580aa7dbb38d4cff8f9d1bb8983f99ef => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:17:35.201398 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6a5ef2-434a-4054-a193-70805f1fbb3e None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:35.201690 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6a5ef2-434a-4054-a193-70805f1fbb3e None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:35.201971 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6a5ef2-434a-4054-a193-70805f1fbb3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:35.269423 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-8f6a5ef2-434a-4054-a193-70805f1fbb3e None None] MFA Rules not processed for user `c3b90a10ca674c59a20171b30843a520`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:17:35.271407 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-8f6a5ef2-434a-4054-a193-70805f1fbb3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:17:35.419193 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 998/2003] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:17:35 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:17:35.425920 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170bdbd9-3525-46d0-97a7-29595330aeb9 admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:17:35.474387 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:DELETE=3,keystone:SELECT=94,keystone:INSERT=3 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:17:35.561263 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:17:35.567177 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:17:35.590316 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170bdbd9-3525-46d0-97a7-29595330aeb9 admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:17:35.599864 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61dac57a-ec92-4534-bc01-b560b25e1fda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:17:35.600855 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dac57a-ec92-4534-bc01-b560b25e1fda admin admin] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:35.601145 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dac57a-ec92-4534-bc01-b560b25e1fda admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:35.601428 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dac57a-ec92-4534-bc01-b560b25e1fda admin admin] PATH_INFO: `/v3/projects` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:35.602075 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61dac57a-ec92-4534-bc01-b560b25e1fda admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:17:35.604033 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61dac57a-ec92-4534-bc01-b560b25e1fda admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:17:35.752706 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1006/2004] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Aug 24 06:17:35 2022] POST /identity/v3/projects => generated 385 bytes in 325 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:17:35.752927 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6a5ef2-434a-4054-a193-70805f1fbb3e None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:17:35.776975 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:17:35.784849 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6a5ef2-434a-4054-a193-70805f1fbb3e None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:17:35.958437 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c51816e-4171-4310-a0d8-b18387dda994 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:17:35.959977 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c51816e-4171-4310-a0d8-b18387dda994 admin admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:35.960810 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c51816e-4171-4310-a0d8-b18387dda994 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:35.961395 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c51816e-4171-4310-a0d8-b18387dda994 admin admin] PATH_INFO: `/v3/users` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:35.962585 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c51816e-4171-4310-a0d8-b18387dda994 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:17:35.980822 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c51816e-4171-4310-a0d8-b18387dda994 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:17:36.082677 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:17:36.297663 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 999/2005] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Aug 24 06:17:35 2022] POST /identity/v3/users => generated 299 bytes in 548 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 06:17:36.303591 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61dac57a-ec92-4534-bc01-b560b25e1fda admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:17:36.315923 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61dac57a-ec92-4534-bc01-b560b25e1fda admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:17:36.323478 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53850b0c-e922-4c41-b55a-134dc9179071 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:17:36.324119 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53850b0c-e922-4c41-b55a-134dc9179071 admin admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:36.324411 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53850b0c-e922-4c41-b55a-134dc9179071 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:36.324712 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53850b0c-e922-4c41-b55a-134dc9179071 admin admin] PATH_INFO: `/v3/roles` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:36.326081 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53850b0c-e922-4c41-b55a-134dc9179071 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:17:36.327058 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53850b0c-e922-4c41-b55a-134dc9179071 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:17:36.341299 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1007/2006] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Aug 24 06:17:36 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 06:17:36.356646 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c51816e-4171-4310-a0d8-b18387dda994 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:17:36.403922 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c51816e-4171-4310-a0d8-b18387dda994 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:17:36.417768 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284f9a91-2c60-413b-9efc-4d8277cfd48d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:17:36.429559 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284f9a91-2c60-413b-9efc-4d8277cfd48d admin admin] REQUEST_METHOD: `PUT` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:36.430824 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284f9a91-2c60-413b-9efc-4d8277cfd48d admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:36.432335 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284f9a91-2c60-413b-9efc-4d8277cfd48d admin admin] PATH_INFO: `/v3/projects/56b1757ce49c40afa993d692f279f729/users/af357606dd7c4ef39dcc1654755fdbb9/roles/49e47cd157d743f6905f1db07ec878a7` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:36.469370 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284f9a91-2c60-413b-9efc-4d8277cfd48d admin admin] RBAC: Authorizing `identity:create_grant(project_id=56b1757ce49c40afa993d692f279f729, user_id=af357606dd7c4ef39dcc1654755fdbb9, role_id=49e47cd157d743f6905f1db07ec878a7)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:17:36.475215 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284f9a91-2c60-413b-9efc-4d8277cfd48d admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:17:36.600977 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1000/2007] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Aug 24 06:17:36 2022] PUT /identity/v3/projects/56b1757ce49c40afa993d692f279f729/users/af357606dd7c4ef39dcc1654755fdbb9/roles/49e47cd157d743f6905f1db07ec878a7 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:17:36.603863 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:17:36.608275 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53850b0c-e922-4c41-b55a-134dc9179071 admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:17:36.629890 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53850b0c-e922-4c41-b55a-134dc9179071 admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:17:36.797909 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eff644d-235a-4283-91c7-2547fbf4ad79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:17:36.800341 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eff644d-235a-4283-91c7-2547fbf4ad79 admin admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:36.801760 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eff644d-235a-4283-91c7-2547fbf4ad79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:36.803509 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eff644d-235a-4283-91c7-2547fbf4ad79 admin admin] PATH_INFO: `/v3/roles` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:36.805526 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eff644d-235a-4283-91c7-2547fbf4ad79 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:17:36.807420 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eff644d-235a-4283-91c7-2547fbf4ad79 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:17:36.838958 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1008/2008] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Aug 24 06:17:36 2022] GET /identity/v3/roles => generated 1420 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:17:36.844844 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284f9a91-2c60-413b-9efc-4d8277cfd48d admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:17:36.866684 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284f9a91-2c60-413b-9efc-4d8277cfd48d admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:17:36.877494 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692c24f1-3573-41b1-8e04-9d74df1ab614 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:17:36.879416 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692c24f1-3573-41b1-8e04-9d74df1ab614 admin admin] REQUEST_METHOD: `PUT` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:36.879884 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692c24f1-3573-41b1-8e04-9d74df1ab614 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:36.880335 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692c24f1-3573-41b1-8e04-9d74df1ab614 admin admin] PATH_INFO: `/v3/projects/56b1757ce49c40afa993d692f279f729/users/af357606dd7c4ef39dcc1654755fdbb9/roles/49e47cd157d743f6905f1db07ec878a7` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:36.888004 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692c24f1-3573-41b1-8e04-9d74df1ab614 admin admin] RBAC: Authorizing `identity:create_grant(project_id=56b1757ce49c40afa993d692f279f729, user_id=af357606dd7c4ef39dcc1654755fdbb9, role_id=49e47cd157d743f6905f1db07ec878a7)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:17:36.892391 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692c24f1-3573-41b1-8e04-9d74df1ab614 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:17:36.927812 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1001/2009] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Aug 24 06:17:36 2022] PUT /identity/v3/projects/56b1757ce49c40afa993d692f279f729/users/af357606dd7c4ef39dcc1654755fdbb9/roles/49e47cd157d743f6905f1db07ec878a7 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 06:17:36.950213 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adae982c-db15-4c3d-8ec1-bd446c55bd8c None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:36.951014 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adae982c-db15-4c3d-8ec1-bd446c55bd8c None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:36.951217 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adae982c-db15-4c3d-8ec1-bd446c55bd8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:37.053120 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-adae982c-db15-4c3d-8ec1-bd446c55bd8c None None] MFA Rules not processed for user `af357606dd7c4ef39dcc1654755fdbb9`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:17:37.058096 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-adae982c-db15-4c3d-8ec1-bd446c55bd8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:17:37.212123 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1009/2010] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:17:36 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:17:37.228258 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692c24f1-3573-41b1-8e04-9d74df1ab614 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:17:37.230946 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-692c24f1-3573-41b1-8e04-9d74df1ab614 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:17:37.304206 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692c24f1-3573-41b1-8e04-9d74df1ab614 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:17:37.321021 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8429394-769e-470f-b90f-69e2ae484d12 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c4def55ac0e48b29836bc5c64bc7402', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:17:37.323424 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8429394-769e-470f-b90f-69e2ae484d12 service cinder] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:37.323424 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8429394-769e-470f-b90f-69e2ae484d12 service cinder] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:37.323766 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8429394-769e-470f-b90f-69e2ae484d12 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:37.381187 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:17:37.476834 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8429394-769e-470f-b90f-69e2ae484d12 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:17:37.480337 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8429394-769e-470f-b90f-69e2ae484d12 service cinder] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:17:37.501166 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1002/2011] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:17:37 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 273 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:17:39.254100 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adae982c-db15-4c3d-8ec1-bd446c55bd8c None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:17:39.256530 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-adae982c-db15-4c3d-8ec1-bd446c55bd8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:17:39.315049 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:17:39.352108 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adae982c-db15-4c3d-8ec1-bd446c55bd8c None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:17:39.356884 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8baa8ea8-6718-464a-9c54-f5845b28e64b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:17:39.357869 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8baa8ea8-6718-464a-9c54-f5845b28e64b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:39.358410 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8baa8ea8-6718-464a-9c54-f5845b28e64b admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:39.358977 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8baa8ea8-6718-464a-9c54-f5845b28e64b admin admin] PATH_INFO: `/v3/users/fc432a6e990541a9a656a1c70c0183a5` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:39.360284 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8baa8ea8-6718-464a-9c54-f5845b28e64b admin admin] RBAC: Authorizing `identity:delete_user(user_id=fc432a6e990541a9a656a1c70c0183a5)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:17:39.363120 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8baa8ea8-6718-464a-9c54-f5845b28e64b admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:17:39.596471 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-8baa8ea8-6718-464a-9c54-f5845b28e64b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fc432a6e990541a9a656a1c70c0183a5'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:17:39.604381 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-8baa8ea8-6718-464a-9c54-f5845b28e64b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fc432a6e990541a9a656a1c70c0183a5'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:17:39.605172 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-8baa8ea8-6718-464a-9c54-f5845b28e64b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fc432a6e990541a9a656a1c70c0183a5'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:17:39.605796 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-8baa8ea8-6718-464a-9c54-f5845b28e64b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fc432a6e990541a9a656a1c70c0183a5'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:17:39.697076 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-8baa8ea8-6718-464a-9c54-f5845b28e64b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fc432a6e990541a9a656a1c70c0183a5'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:17:39.755695 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1010/2012] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Aug 24 06:17:39 2022] DELETE /identity/v3/users/fc432a6e990541a9a656a1c70c0183a5 => generated 0 bytes in 500 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:17:39.765038 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef98a71-8ee9-4635-aeb3-42714dac2de2 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:39.765951 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef98a71-8ee9-4635-aeb3-42714dac2de2 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:39.766380 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef98a71-8ee9-4635-aeb3-42714dac2de2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:39.795037 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-8ef98a71-8ee9-4635-aeb3-42714dac2de2 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:17:39.797237 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-8ef98a71-8ee9-4635-aeb3-42714dac2de2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:17:39.901829 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1003/2013] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:17:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:17:39.912303 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8baa8ea8-6718-464a-9c54-f5845b28e64b admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:17:39.914608 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-8baa8ea8-6718-464a-9c54-f5845b28e64b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:17:40.018507 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8baa8ea8-6718-464a-9c54-f5845b28e64b admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:17:40.043956 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2448482e-e281-4f78-ab9b-89b765fa23fc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bb6c79db75e4d169331c02133d1ec32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:17:40.043956 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2448482e-e281-4f78-ab9b-89b765fa23fc service neutron] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:40.043956 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2448482e-e281-4f78-ab9b-89b765fa23fc service neutron] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:40.043956 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2448482e-e281-4f78-ab9b-89b765fa23fc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:40.060729 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2448482e-e281-4f78-ab9b-89b765fa23fc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:17:40.070088 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2448482e-e281-4f78-ab9b-89b765fa23fc service neutron] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:17:40.093250 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1011/2014] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:17:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:17:41.037902 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef98a71-8ee9-4635-aeb3-42714dac2de2 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:17:41.043975 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-8ef98a71-8ee9-4635-aeb3-42714dac2de2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:17:41.090653 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef98a71-8ee9-4635-aeb3-42714dac2de2 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:17:41.098251 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7b7374-75fd-4481-b5c3-32b3c5451570 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:17:41.098969 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7b7374-75fd-4481-b5c3-32b3c5451570 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:41.099402 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7b7374-75fd-4481-b5c3-32b3c5451570 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:41.099811 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7b7374-75fd-4481-b5c3-32b3c5451570 admin admin] PATH_INFO: `/v3/projects/21a786448e1c40d98216c6d6e2c9ada0` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:41.101091 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7b7374-75fd-4481-b5c3-32b3c5451570 admin admin] RBAC: Authorizing `identity:delete_project(project_id=21a786448e1c40d98216c6d6e2c9ada0)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:17:41.103089 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7b7374-75fd-4481-b5c3-32b3c5451570 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:17:41.235104 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-7b7b7374-75fd-4481-b5c3-32b3c5451570 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '21a786448e1c40d98216c6d6e2c9ada0', 'request_id': 'req-7b7b7374-75fd-4481-b5c3-32b3c5451570'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:17:41.248418 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-7b7b7374-75fd-4481-b5c3-32b3c5451570 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '21a786448e1c40d98216c6d6e2c9ada0', 'request_id': 'req-7b7b7374-75fd-4481-b5c3-32b3c5451570'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:17:41.258218 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-7b7b7374-75fd-4481-b5c3-32b3c5451570 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '21a786448e1c40d98216c6d6e2c9ada0', 'request_id': 'req-7b7b7374-75fd-4481-b5c3-32b3c5451570'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:17:41.350705 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-7b7b7374-75fd-4481-b5c3-32b3c5451570 admin admin] The token cache is being invalidate because project 21a786448e1c40d98216c6d6e2c9ada0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81852) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:17:41.351269 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-7b7b7374-75fd-4481-b5c3-32b3c5451570 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:17:41.352043 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-7b7b7374-75fd-4481-b5c3-32b3c5451570 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:17:41.354397 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1004/2015] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Aug 24 06:17:41 2022] DELETE /identity/v3/projects/21a786448e1c40d98216c6d6e2c9ada0 => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:17:41.359438 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2448482e-e281-4f78-ab9b-89b765fa23fc service neutron] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:17:41.371171 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2448482e-e281-4f78-ab9b-89b765fa23fc service neutron] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:17:41.460735 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:17:41.467397 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c025755a-40f6-4cc3-980c-6a1f997d9e50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:17:41.468032 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c025755a-40f6-4cc3-980c-6a1f997d9e50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:41.468315 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c025755a-40f6-4cc3-980c-6a1f997d9e50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:41.474079 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c025755a-40f6-4cc3-980c-6a1f997d9e50 admin admin] PATH_INFO: `/v3/users/ee36e0ddf7a141b6a03bed0dc09ab3c9` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:41.474079 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c025755a-40f6-4cc3-980c-6a1f997d9e50 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ee36e0ddf7a141b6a03bed0dc09ab3c9)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:17:41.474708 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c025755a-40f6-4cc3-980c-6a1f997d9e50 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:17:41.686241 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-c025755a-40f6-4cc3-980c-6a1f997d9e50 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ee36e0ddf7a141b6a03bed0dc09ab3c9'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:17:41.694379 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-c025755a-40f6-4cc3-980c-6a1f997d9e50 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ee36e0ddf7a141b6a03bed0dc09ab3c9'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:17:41.695264 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-c025755a-40f6-4cc3-980c-6a1f997d9e50 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ee36e0ddf7a141b6a03bed0dc09ab3c9'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:17:41.696013 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-c025755a-40f6-4cc3-980c-6a1f997d9e50 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ee36e0ddf7a141b6a03bed0dc09ab3c9'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:17:41.805051 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-c025755a-40f6-4cc3-980c-6a1f997d9e50 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ee36e0ddf7a141b6a03bed0dc09ab3c9'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:17:41.830825 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1012/2016] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Aug 24 06:17:41 2022] DELETE /identity/v3/users/ee36e0ddf7a141b6a03bed0dc09ab3c9 => generated 0 bytes in 473 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:17:42.335849 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7b7374-75fd-4481-b5c3-32b3c5451570 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:17:42.338412 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-7b7b7374-75fd-4481-b5c3-32b3c5451570 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:17:42.355487 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7b7374-75fd-4481-b5c3-32b3c5451570 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:17:42.448806 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:17:42.468346 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de18b73e-9356-454e-8f00-3e090776003b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:17:42.468899 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de18b73e-9356-454e-8f00-3e090776003b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:42.469124 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de18b73e-9356-454e-8f00-3e090776003b admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:42.469438 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de18b73e-9356-454e-8f00-3e090776003b admin admin] PATH_INFO: `/v3/projects/d914df5358d64502be8cd74a8a653003` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:42.471306 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de18b73e-9356-454e-8f00-3e090776003b admin admin] RBAC: Authorizing `identity:delete_project(project_id=d914df5358d64502be8cd74a8a653003)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:17:42.473816 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de18b73e-9356-454e-8f00-3e090776003b admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:17:42.588423 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-de18b73e-9356-454e-8f00-3e090776003b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd914df5358d64502be8cd74a8a653003', 'request_id': 'req-de18b73e-9356-454e-8f00-3e090776003b'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:17:42.595039 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-de18b73e-9356-454e-8f00-3e090776003b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd914df5358d64502be8cd74a8a653003', 'request_id': 'req-de18b73e-9356-454e-8f00-3e090776003b'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:17:42.605125 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-de18b73e-9356-454e-8f00-3e090776003b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd914df5358d64502be8cd74a8a653003', 'request_id': 'req-de18b73e-9356-454e-8f00-3e090776003b'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:17:42.698220 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-de18b73e-9356-454e-8f00-3e090776003b admin admin] The token cache is being invalidate because project d914df5358d64502be8cd74a8a653003 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81852) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:17:42.699508 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-de18b73e-9356-454e-8f00-3e090776003b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:17:42.700401 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-de18b73e-9356-454e-8f00-3e090776003b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:17:42.703527 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1005/2017] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Aug 24 06:17:42 2022] DELETE /identity/v3/projects/d914df5358d64502be8cd74a8a653003 => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:17:42.714704 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac7090e-866f-4f69-8d77-82e02a99f809 None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:42.715245 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac7090e-866f-4f69-8d77-82e02a99f809 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:42.715245 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac7090e-866f-4f69-8d77-82e02a99f809 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:42.744760 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-dac7090e-866f-4f69-8d77-82e02a99f809 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:17:42.745982 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-dac7090e-866f-4f69-8d77-82e02a99f809 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:17:42.825623 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1013/2018] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:17:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:17:42.834303 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37eed271-4a95-438c-bee7-e2c04a80074b None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:42.834575 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37eed271-4a95-438c-bee7-e2c04a80074b None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:42.834851 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37eed271-4a95-438c-bee7-e2c04a80074b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:42.860136 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-37eed271-4a95-438c-bee7-e2c04a80074b None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:17:42.862035 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-37eed271-4a95-438c-bee7-e2c04a80074b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:17:42.870097 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1006/2019] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:17:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:17:42.876190 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac7090e-866f-4f69-8d77-82e02a99f809 None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:17:42.886723 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac7090e-866f-4f69-8d77-82e02a99f809 None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:17:42.892253 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe4fd41-f341-4ee0-b2ad-9fa56176eb90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:17:42.892940 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe4fd41-f341-4ee0-b2ad-9fa56176eb90 admin admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:42.893220 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe4fd41-f341-4ee0-b2ad-9fa56176eb90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:42.893489 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe4fd41-f341-4ee0-b2ad-9fa56176eb90 admin admin] PATH_INFO: `/v3/domains` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:42.894136 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe4fd41-f341-4ee0-b2ad-9fa56176eb90 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:17:42.894461 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe4fd41-f341-4ee0-b2ad-9fa56176eb90 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:17:42.896060 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe4fd41-f341-4ee0-b2ad-9fa56176eb90 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:17:42.910119 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1014/2020] 10.222.0.62 () {58 vars in 1153 bytes} [Wed Aug 24 06:17:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:17:42.914270 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37eed271-4a95-438c-bee7-e2c04a80074b None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:17:42.937451 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37eed271-4a95-438c-bee7-e2c04a80074b None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:17:42.945134 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42dbb5eb-aca6-45a7-ad01-6c242785a775 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:17:42.946296 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dbb5eb-aca6-45a7-ad01-6c242785a775 admin admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:42.946589 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dbb5eb-aca6-45a7-ad01-6c242785a775 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:42.946933 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dbb5eb-aca6-45a7-ad01-6c242785a775 admin admin] PATH_INFO: `/v3/projects` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:42.947647 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42dbb5eb-aca6-45a7-ad01-6c242785a775 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:17:42.949750 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42dbb5eb-aca6-45a7-ad01-6c242785a775 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:17:43.060063 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1007/2021] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Aug 24 06:17:42 2022] POST /identity/v3/projects => generated 385 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:17:43.064004 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe4fd41-f341-4ee0-b2ad-9fa56176eb90 admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:17:43.077584 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe4fd41-f341-4ee0-b2ad-9fa56176eb90 admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:17:43.155493 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af7daed-9d08-4acb-a698-ff20b7f66e8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:17:43.156870 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af7daed-9d08-4acb-a698-ff20b7f66e8b admin admin] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:43.156870 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af7daed-9d08-4acb-a698-ff20b7f66e8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:43.156870 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af7daed-9d08-4acb-a698-ff20b7f66e8b admin admin] PATH_INFO: `/v3/users` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:43.157294 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af7daed-9d08-4acb-a698-ff20b7f66e8b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:17:43.159530 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af7daed-9d08-4acb-a698-ff20b7f66e8b admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:17:43.172388 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:17:43.245884 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1015/2022] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Aug 24 06:17:43 2022] POST /identity/v3/users => generated 293 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:17:43.249999 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42dbb5eb-aca6-45a7-ad01-6c242785a775 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:17:43.261366 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42dbb5eb-aca6-45a7-ad01-6c242785a775 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:17:43.268871 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e87b36d-1151-4d94-9f33-f81ec114de41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:17:43.269874 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e87b36d-1151-4d94-9f33-f81ec114de41 admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:43.270345 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e87b36d-1151-4d94-9f33-f81ec114de41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:43.270873 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e87b36d-1151-4d94-9f33-f81ec114de41 admin admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:43.271535 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e87b36d-1151-4d94-9f33-f81ec114de41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:17:43.273540 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e87b36d-1151-4d94-9f33-f81ec114de41 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:17:43.286525 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1008/2023] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Aug 24 06:17:43 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:17:43.290247 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af7daed-9d08-4acb-a698-ff20b7f66e8b admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:17:43.303445 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af7daed-9d08-4acb-a698-ff20b7f66e8b admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:17:43.308780 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cbad98-f630-4586-9fa2-47169a61f3f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:17:43.311984 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cbad98-f630-4586-9fa2-47169a61f3f1 admin admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:43.311984 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cbad98-f630-4586-9fa2-47169a61f3f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:43.311984 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cbad98-f630-4586-9fa2-47169a61f3f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:43.311984 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cbad98-f630-4586-9fa2-47169a61f3f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:17:43.312411 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cbad98-f630-4586-9fa2-47169a61f3f1 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:17:43.325162 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1016/2024] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Aug 24 06:17:43 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:17:43.329098 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e87b36d-1151-4d94-9f33-f81ec114de41 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:17:43.344939 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e87b36d-1151-4d94-9f33-f81ec114de41 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:17:43.350986 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4304bf94-7458-4341-b62d-a96c2654a5fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:17:43.351671 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4304bf94-7458-4341-b62d-a96c2654a5fc admin admin] REQUEST_METHOD: `PUT` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:43.351952 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4304bf94-7458-4341-b62d-a96c2654a5fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:43.352277 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4304bf94-7458-4341-b62d-a96c2654a5fc admin admin] PATH_INFO: `/v3/projects/f77eb66e8196436cb6c7ce39c91e7e5f/users/8599c2c67689439daa2ec5cd38aea03d/roles/580aa7dbb38d4cff8f9d1bb8983f99ef` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:43.376960 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4304bf94-7458-4341-b62d-a96c2654a5fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=f77eb66e8196436cb6c7ce39c91e7e5f, user_id=8599c2c67689439daa2ec5cd38aea03d, role_id=580aa7dbb38d4cff8f9d1bb8983f99ef)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:17:43.379625 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4304bf94-7458-4341-b62d-a96c2654a5fc admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:17:43.448317 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1009/2025] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Aug 24 06:17:43 2022] PUT /identity/v3/projects/f77eb66e8196436cb6c7ce39c91e7e5f/users/8599c2c67689439daa2ec5cd38aea03d/roles/580aa7dbb38d4cff8f9d1bb8983f99ef => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 06:17:43.459435 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a70f17d-2204-44b5-800e-cfea336d0f0b None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:43.459718 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a70f17d-2204-44b5-800e-cfea336d0f0b None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:43.459993 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a70f17d-2204-44b5-800e-cfea336d0f0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:43.496814 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-5a70f17d-2204-44b5-800e-cfea336d0f0b None None] MFA Rules not processed for user `8599c2c67689439daa2ec5cd38aea03d`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:17:43.498940 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-5a70f17d-2204-44b5-800e-cfea336d0f0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:17:43.607624 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1017/2026] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:17:43 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:17:43.619932 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4304bf94-7458-4341-b62d-a96c2654a5fc admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:17:43.621812 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-4304bf94-7458-4341-b62d-a96c2654a5fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:17:43.697756 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4304bf94-7458-4341-b62d-a96c2654a5fc admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:17:43.701367 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5511f026-a0dd-4db5-a2b7-7069fe77d3e7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c4def55ac0e48b29836bc5c64bc7402', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:17:43.701974 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5511f026-a0dd-4db5-a2b7-7069fe77d3e7 service cinder] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:43.702392 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5511f026-a0dd-4db5-a2b7-7069fe77d3e7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:43.702720 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5511f026-a0dd-4db5-a2b7-7069fe77d3e7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:43.716730 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5511f026-a0dd-4db5-a2b7-7069fe77d3e7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:17:43.718316 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5511f026-a0dd-4db5-a2b7-7069fe77d3e7 service cinder] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:17:43.729352 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1010/2027] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:17:43 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:17:43.749711 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a70f17d-2204-44b5-800e-cfea336d0f0b None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:17:43.824728 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a70f17d-2204-44b5-800e-cfea336d0f0b None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:17:43.830262 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c33345-5929-4160-8cdf-b262971c3fb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:17:43.831188 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c33345-5929-4160-8cdf-b262971c3fb0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:43.831559 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c33345-5929-4160-8cdf-b262971c3fb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:43.831985 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c33345-5929-4160-8cdf-b262971c3fb0 admin admin] PATH_INFO: `/v3/users/8599c2c67689439daa2ec5cd38aea03d` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:43.833535 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c33345-5929-4160-8cdf-b262971c3fb0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8599c2c67689439daa2ec5cd38aea03d)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:17:43.835843 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c33345-5929-4160-8cdf-b262971c3fb0 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:17:44.357463 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-b2c33345-5929-4160-8cdf-b262971c3fb0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8599c2c67689439daa2ec5cd38aea03d'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:17:44.365857 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-b2c33345-5929-4160-8cdf-b262971c3fb0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8599c2c67689439daa2ec5cd38aea03d'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:17:44.366691 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-b2c33345-5929-4160-8cdf-b262971c3fb0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8599c2c67689439daa2ec5cd38aea03d'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:17:44.367376 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-b2c33345-5929-4160-8cdf-b262971c3fb0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8599c2c67689439daa2ec5cd38aea03d'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:17:44.444756 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-b2c33345-5929-4160-8cdf-b262971c3fb0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8599c2c67689439daa2ec5cd38aea03d'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:17:44.454544 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:INSERT=2,keystone:SELECT=89,keystone:DELETE=9 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:17:44.467500 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1018/2028] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Aug 24 06:17:43 2022] DELETE /identity/v3/users/8599c2c67689439daa2ec5cd38aea03d => generated 0 bytes in 718 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:17:44.472305 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414c0c26-4da5-4a2a-8068-057f585dc59a None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:44.472869 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414c0c26-4da5-4a2a-8068-057f585dc59a None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:44.473216 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414c0c26-4da5-4a2a-8068-057f585dc59a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:44.496476 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-414c0c26-4da5-4a2a-8068-057f585dc59a None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:17:44.497854 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-414c0c26-4da5-4a2a-8068-057f585dc59a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:17:44.514952 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:17:44.574660 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1011/2029] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:17:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:17:44.585608 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c33345-5929-4160-8cdf-b262971c3fb0 admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:17:44.587099 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-b2c33345-5929-4160-8cdf-b262971c3fb0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:17:44.649490 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c33345-5929-4160-8cdf-b262971c3fb0 admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:17:44.653083 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f226cc-840e-4e99-b5d8-b289a3bdfdd7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bb6c79db75e4d169331c02133d1ec32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:17:44.653692 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f226cc-840e-4e99-b5d8-b289a3bdfdd7 service neutron] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:44.653974 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f226cc-840e-4e99-b5d8-b289a3bdfdd7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:44.654272 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f226cc-840e-4e99-b5d8-b289a3bdfdd7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:44.663728 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f226cc-840e-4e99-b5d8-b289a3bdfdd7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:17:44.665203 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f226cc-840e-4e99-b5d8-b289a3bdfdd7 service neutron] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:17:44.676557 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1019/2030] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:17:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:17:45.293209 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414c0c26-4da5-4a2a-8068-057f585dc59a None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:17:45.294694 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-414c0c26-4da5-4a2a-8068-057f585dc59a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:17:45.301394 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968a8bfe-2724-4b38-9fed-7e4c46ad4fca None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:45.301872 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968a8bfe-2724-4b38-9fed-7e4c46ad4fca None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:45.302040 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968a8bfe-2724-4b38-9fed-7e4c46ad4fca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:45.312881 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414c0c26-4da5-4a2a-8068-057f585dc59a None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:17:45.317594 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16885b0-b620-4eae-b90a-c2027490804d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:17:45.318252 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16885b0-b620-4eae-b90a-c2027490804d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:45.318578 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16885b0-b620-4eae-b90a-c2027490804d admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:45.318847 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16885b0-b620-4eae-b90a-c2027490804d admin admin] PATH_INFO: `/v3/projects/f77eb66e8196436cb6c7ce39c91e7e5f` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:45.319983 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16885b0-b620-4eae-b90a-c2027490804d admin admin] RBAC: Authorizing `identity:delete_project(project_id=f77eb66e8196436cb6c7ce39c91e7e5f)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:17:45.323096 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16885b0-b620-4eae-b90a-c2027490804d admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:17:45.324231 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-968a8bfe-2724-4b38-9fed-7e4c46ad4fca None None] MFA Rules not processed for user `d3a03b5f82724f37a2134554d42b2925`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:17:45.325488 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-968a8bfe-2724-4b38-9fed-7e4c46ad4fca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:17:45.383066 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1020/2031] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:17:45 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 06:17:45.429393 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-b16885b0-b620-4eae-b90a-c2027490804d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f77eb66e8196436cb6c7ce39c91e7e5f', 'request_id': 'req-b16885b0-b620-4eae-b90a-c2027490804d'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:17:45.435431 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-b16885b0-b620-4eae-b90a-c2027490804d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f77eb66e8196436cb6c7ce39c91e7e5f', 'request_id': 'req-b16885b0-b620-4eae-b90a-c2027490804d'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:17:45.441815 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-b16885b0-b620-4eae-b90a-c2027490804d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f77eb66e8196436cb6c7ce39c91e7e5f', 'request_id': 'req-b16885b0-b620-4eae-b90a-c2027490804d'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:17:45.507356 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-b16885b0-b620-4eae-b90a-c2027490804d admin admin] The token cache is being invalidate because project f77eb66e8196436cb6c7ce39c91e7e5f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81852) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:17:45.508009 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-b16885b0-b620-4eae-b90a-c2027490804d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:17:45.508258 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-b16885b0-b620-4eae-b90a-c2027490804d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:17:45.509768 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1012/2032] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Aug 24 06:17:45 2022] DELETE /identity/v3/projects/f77eb66e8196436cb6c7ce39c91e7e5f => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:17:45.514641 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2da232-35c5-4245-868e-cfab05248981 None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:45.514916 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2da232-35c5-4245-868e-cfab05248981 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:45.521523 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2da232-35c5-4245-868e-cfab05248981 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:45.534992 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-df2da232-35c5-4245-868e-cfab05248981 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:17:45.536188 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-df2da232-35c5-4245-868e-cfab05248981 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:17:45.607103 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1021/2033] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:17:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 06:17:45.615375 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2885cfa-318c-4212-b316-87db134c59b3 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:45.615939 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2885cfa-318c-4212-b316-87db134c59b3 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:45.616216 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2885cfa-318c-4212-b316-87db134c59b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:45.639024 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-b2885cfa-318c-4212-b316-87db134c59b3 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:17:45.640240 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-b2885cfa-318c-4212-b316-87db134c59b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:17:45.648252 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1013/2034] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:17:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:17:45.652696 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2da232-35c5-4245-868e-cfab05248981 None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:17:45.662989 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2da232-35c5-4245-868e-cfab05248981 None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:17:45.667789 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a175fca-df31-4043-b3a3-a17580931f2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:17:45.668463 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a175fca-df31-4043-b3a3-a17580931f2c admin admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:45.668840 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a175fca-df31-4043-b3a3-a17580931f2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:45.669180 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a175fca-df31-4043-b3a3-a17580931f2c admin admin] PATH_INFO: `/v3/domains` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:45.669895 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a175fca-df31-4043-b3a3-a17580931f2c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:17:45.670267 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a175fca-df31-4043-b3a3-a17580931f2c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:17:45.671942 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a175fca-df31-4043-b3a3-a17580931f2c admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:17:45.684483 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1022/2035] 10.222.0.62 () {58 vars in 1153 bytes} [Wed Aug 24 06:17:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 06:17:45.691045 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2885cfa-318c-4212-b316-87db134c59b3 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:17:45.700714 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2885cfa-318c-4212-b316-87db134c59b3 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:17:45.705961 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9359f0a-d791-4eef-b1f6-fd7a758051c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:17:45.706791 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9359f0a-d791-4eef-b1f6-fd7a758051c7 admin admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:45.707074 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9359f0a-d791-4eef-b1f6-fd7a758051c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:45.707405 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9359f0a-d791-4eef-b1f6-fd7a758051c7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:45.708032 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9359f0a-d791-4eef-b1f6-fd7a758051c7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:17:45.710132 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9359f0a-d791-4eef-b1f6-fd7a758051c7 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:17:45.836076 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1014/2036] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Aug 24 06:17:45 2022] POST /identity/v3/projects => generated 375 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:17:45.840479 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a175fca-df31-4043-b3a3-a17580931f2c admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:17:45.850973 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a175fca-df31-4043-b3a3-a17580931f2c admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:17:45.876933 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:17:45.911676 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052c13d0-db6d-44ff-8a42-fb327f91d335 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:17:45.912257 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052c13d0-db6d-44ff-8a42-fb327f91d335 admin admin] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:45.912599 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052c13d0-db6d-44ff-8a42-fb327f91d335 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:45.912928 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052c13d0-db6d-44ff-8a42-fb327f91d335 admin admin] PATH_INFO: `/v3/users` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:45.914710 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052c13d0-db6d-44ff-8a42-fb327f91d335 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:17:45.915668 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052c13d0-db6d-44ff-8a42-fb327f91d335 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:17:45.983266 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1023/2037] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Aug 24 06:17:45 2022] POST /identity/v3/users => generated 288 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:17:45.987887 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9359f0a-d791-4eef-b1f6-fd7a758051c7 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:17:45.998563 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9359f0a-d791-4eef-b1f6-fd7a758051c7 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:17:46.004012 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26607139-f48a-42f5-bb4e-41ad69791d7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:17:46.004645 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26607139-f48a-42f5-bb4e-41ad69791d7b admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:46.004921 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26607139-f48a-42f5-bb4e-41ad69791d7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:46.005464 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26607139-f48a-42f5-bb4e-41ad69791d7b admin admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:46.005821 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26607139-f48a-42f5-bb4e-41ad69791d7b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:17:46.007523 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26607139-f48a-42f5-bb4e-41ad69791d7b admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:17:46.018872 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1015/2038] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Aug 24 06:17:45 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:17:46.023045 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052c13d0-db6d-44ff-8a42-fb327f91d335 admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:17:46.042081 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052c13d0-db6d-44ff-8a42-fb327f91d335 admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:17:46.047115 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5569c8-98ba-4531-b60d-0a53b6e84e73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:17:46.047734 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5569c8-98ba-4531-b60d-0a53b6e84e73 admin admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:46.048015 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5569c8-98ba-4531-b60d-0a53b6e84e73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:46.048835 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5569c8-98ba-4531-b60d-0a53b6e84e73 admin admin] PATH_INFO: `/v3/roles` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:46.049516 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5569c8-98ba-4531-b60d-0a53b6e84e73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:17:46.051363 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5569c8-98ba-4531-b60d-0a53b6e84e73 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:17:46.062476 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1024/2039] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Aug 24 06:17:46 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:17:46.067130 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26607139-f48a-42f5-bb4e-41ad69791d7b admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:17:46.079044 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26607139-f48a-42f5-bb4e-41ad69791d7b admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:17:46.087473 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26aecf27-6925-4eec-92c2-8d8757405669 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:17:46.088666 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26aecf27-6925-4eec-92c2-8d8757405669 admin admin] REQUEST_METHOD: `PUT` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:46.088666 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26aecf27-6925-4eec-92c2-8d8757405669 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:46.088843 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26aecf27-6925-4eec-92c2-8d8757405669 admin admin] PATH_INFO: `/v3/projects/066a4ae88895464980e4bd1bbf0b25c4/users/ef76e0e1496946e5a5927a452069c90d/roles/580aa7dbb38d4cff8f9d1bb8983f99ef` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:46.105387 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26aecf27-6925-4eec-92c2-8d8757405669 admin admin] RBAC: Authorizing `identity:create_grant(project_id=066a4ae88895464980e4bd1bbf0b25c4, user_id=ef76e0e1496946e5a5927a452069c90d, role_id=580aa7dbb38d4cff8f9d1bb8983f99ef)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:17:46.108539 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26aecf27-6925-4eec-92c2-8d8757405669 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:17:46.233117 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1016/2040] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Aug 24 06:17:46 2022] PUT /identity/v3/projects/066a4ae88895464980e4bd1bbf0b25c4/users/ef76e0e1496946e5a5927a452069c90d/roles/580aa7dbb38d4cff8f9d1bb8983f99ef => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:17:46.244146 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70b2964-f680-4b53-8fff-5689e74ebf54 None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:46.244448 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70b2964-f680-4b53-8fff-5689e74ebf54 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:46.244762 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70b2964-f680-4b53-8fff-5689e74ebf54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:46.273965 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-e70b2964-f680-4b53-8fff-5689e74ebf54 None None] MFA Rules not processed for user `ef76e0e1496946e5a5927a452069c90d`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:17:46.276287 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-e70b2964-f680-4b53-8fff-5689e74ebf54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:17:46.285583 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26aecf27-6925-4eec-92c2-8d8757405669 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:17:46.287433 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-26aecf27-6925-4eec-92c2-8d8757405669 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:17:46.333383 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:17:46.350972 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26aecf27-6925-4eec-92c2-8d8757405669 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:17:46.351655 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1025/2041] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:17:46 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 06:17:46.354666 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86132f15-30b0-466c-848a-505d69676f2b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c4def55ac0e48b29836bc5c64bc7402', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:17:46.357113 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86132f15-30b0-466c-848a-505d69676f2b service cinder] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:46.357464 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86132f15-30b0-466c-848a-505d69676f2b service cinder] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:46.357801 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86132f15-30b0-466c-848a-505d69676f2b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:46.359648 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-86132f15-30b0-466c-848a-505d69676f2b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:17:46.362192 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70b2964-f680-4b53-8fff-5689e74ebf54 None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:17:46.363594 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-e70b2964-f680-4b53-8fff-5689e74ebf54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:17:46.375067 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70b2964-f680-4b53-8fff-5689e74ebf54 None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:17:46.379266 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d987676e-ca5e-4faa-a9c3-a451977eaebe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c4def55ac0e48b29836bc5c64bc7402', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:17:46.380032 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d987676e-ca5e-4faa-a9c3-a451977eaebe service cinder] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:46.380272 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d987676e-ca5e-4faa-a9c3-a451977eaebe service cinder] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:46.380475 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d987676e-ca5e-4faa-a9c3-a451977eaebe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:46.389793 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d987676e-ca5e-4faa-a9c3-a451977eaebe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:17:46.391335 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d987676e-ca5e-4faa-a9c3-a451977eaebe service cinder] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:17:46.401047 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1026/2042] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:17:46 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 06:17:46.430452 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86132f15-30b0-466c-848a-505d69676f2b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:17:46.431887 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86132f15-30b0-466c-848a-505d69676f2b service cinder] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:17:46.441394 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1017/2043] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:17:46 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 06:17:52.496874 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9217960-fb48-4841-bb10-b9cce19b45a2 None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:52.498485 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9217960-fb48-4841-bb10-b9cce19b45a2 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:52.498929 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9217960-fb48-4841-bb10-b9cce19b45a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:52.523120 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-f9217960-fb48-4841-bb10-b9cce19b45a2 None None] MFA Rules not processed for user `b82726a12436435cb931b986ef884c2e`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:17:52.524653 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-f9217960-fb48-4841-bb10-b9cce19b45a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:17:52.632104 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1027/2044] 10.222.0.62 () {58 vars in 1009 bytes} [Wed Aug 24 06:17:52 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:17:52.681646 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86132f15-30b0-466c-848a-505d69676f2b service cinder] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:17:52.681646 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-86132f15-30b0-466c-848a-505d69676f2b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:17:52.723837 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9217960-fb48-4841-bb10-b9cce19b45a2 None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:17:52.846574 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:17:52.887676 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86132f15-30b0-466c-848a-505d69676f2b service cinder] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:17:52.908263 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e048dc8-1261-4d50-8903-197f37d9a41b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14d4fed04fb144d08918bd5d4f808d23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:17:52.908263 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e048dc8-1261-4d50-8903-197f37d9a41b service swift] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:52.908263 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e048dc8-1261-4d50-8903-197f37d9a41b service swift] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:52.908263 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e048dc8-1261-4d50-8903-197f37d9a41b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:52.921302 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e048dc8-1261-4d50-8903-197f37d9a41b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:17:52.925805 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e048dc8-1261-4d50-8903-197f37d9a41b service swift] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:17:52.941555 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9217960-fb48-4841-bb10-b9cce19b45a2 None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:17:52.945646 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1018/2045] 10.222.0.62 () {60 vars in 1384 bytes} [Wed Aug 24 06:17:52 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 268 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 06:17:52.951194 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda5e929-0639-4da5-85b0-2d01df66f7ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:17:52.952003 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda5e929-0639-4da5-85b0-2d01df66f7ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:52.952472 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda5e929-0639-4da5-85b0-2d01df66f7ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:52.953545 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda5e929-0639-4da5-85b0-2d01df66f7ce admin admin] PATH_INFO: `/v3/users/ef76e0e1496946e5a5927a452069c90d` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:52.955575 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda5e929-0639-4da5-85b0-2d01df66f7ce admin admin] RBAC: Authorizing `identity:delete_user(user_id=ef76e0e1496946e5a5927a452069c90d)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:17:52.958484 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda5e929-0639-4da5-85b0-2d01df66f7ce admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:17:53.244599 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-eda5e929-0639-4da5-85b0-2d01df66f7ce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ef76e0e1496946e5a5927a452069c90d'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:17:53.267783 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-eda5e929-0639-4da5-85b0-2d01df66f7ce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ef76e0e1496946e5a5927a452069c90d'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:17:53.271138 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-eda5e929-0639-4da5-85b0-2d01df66f7ce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ef76e0e1496946e5a5927a452069c90d'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:17:53.273657 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-eda5e929-0639-4da5-85b0-2d01df66f7ce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ef76e0e1496946e5a5927a452069c90d'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:17:53.348479 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-eda5e929-0639-4da5-85b0-2d01df66f7ce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ef76e0e1496946e5a5927a452069c90d'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:17:53.374636 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1028/2046] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Aug 24 06:17:52 2022] DELETE /identity/v3/users/ef76e0e1496946e5a5927a452069c90d => generated 0 bytes in 652 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:17:53.380278 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ae74b9-7f34-4829-a708-d92513604423 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:53.380860 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ae74b9-7f34-4829-a708-d92513604423 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:53.381402 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ae74b9-7f34-4829-a708-d92513604423 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:53.406789 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-82ae74b9-7f34-4829-a708-d92513604423 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:17:53.409106 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-82ae74b9-7f34-4829-a708-d92513604423 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:17:53.515593 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1019/2047] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:17:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 06:17:53.533532 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda5e929-0639-4da5-85b0-2d01df66f7ce admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:17:53.534222 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-eda5e929-0639-4da5-85b0-2d01df66f7ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:17:53.631162 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda5e929-0639-4da5-85b0-2d01df66f7ce admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:17:53.637364 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d71cd8-335f-4078-8058-c48f7acbdc0a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bb6c79db75e4d169331c02133d1ec32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:17:53.638645 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d71cd8-335f-4078-8058-c48f7acbdc0a service neutron] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:53.639204 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d71cd8-335f-4078-8058-c48f7acbdc0a service neutron] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:53.639323 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d71cd8-335f-4078-8058-c48f7acbdc0a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:53.654224 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d71cd8-335f-4078-8058-c48f7acbdc0a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:17:53.656869 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d71cd8-335f-4078-8058-c48f7acbdc0a service neutron] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:17:53.671790 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1029/2048] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:17:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:17:54.356729 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ae74b9-7f34-4829-a708-d92513604423 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:17:54.359357 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-82ae74b9-7f34-4829-a708-d92513604423 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:17:54.389096 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ae74b9-7f34-4829-a708-d92513604423 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:17:54.410063 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c9b822-2979-43f5-8162-877ec39d319a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:17:54.411105 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c9b822-2979-43f5-8162-877ec39d319a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:54.411604 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c9b822-2979-43f5-8162-877ec39d319a admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:54.412126 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c9b822-2979-43f5-8162-877ec39d319a admin admin] PATH_INFO: `/v3/projects/066a4ae88895464980e4bd1bbf0b25c4` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:54.413723 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c9b822-2979-43f5-8162-877ec39d319a admin admin] RBAC: Authorizing `identity:delete_project(project_id=066a4ae88895464980e4bd1bbf0b25c4)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:17:54.416649 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c9b822-2979-43f5-8162-877ec39d319a admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:17:54.430629 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:17:54.571778 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-79c9b822-2979-43f5-8162-877ec39d319a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '066a4ae88895464980e4bd1bbf0b25c4', 'request_id': 'req-79c9b822-2979-43f5-8162-877ec39d319a'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:17:54.578956 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-79c9b822-2979-43f5-8162-877ec39d319a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '066a4ae88895464980e4bd1bbf0b25c4', 'request_id': 'req-79c9b822-2979-43f5-8162-877ec39d319a'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:17:54.590922 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-79c9b822-2979-43f5-8162-877ec39d319a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '066a4ae88895464980e4bd1bbf0b25c4', 'request_id': 'req-79c9b822-2979-43f5-8162-877ec39d319a'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:17:54.622262 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d71cd8-335f-4078-8058-c48f7acbdc0a service neutron] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:17:54.623944 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-c2d71cd8-335f-4078-8058-c48f7acbdc0a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:17:54.638647 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d71cd8-335f-4078-8058-c48f7acbdc0a service neutron] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:17:54.666706 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-79c9b822-2979-43f5-8162-877ec39d319a admin admin] The token cache is being invalidate because project 066a4ae88895464980e4bd1bbf0b25c4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81852) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:17:54.666999 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-79c9b822-2979-43f5-8162-877ec39d319a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:17:54.673308 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-79c9b822-2979-43f5-8162-877ec39d319a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:17:54.677385 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1020/2049] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Aug 24 06:17:54 2022] DELETE /identity/v3/projects/066a4ae88895464980e4bd1bbf0b25c4 => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 06:17:54.684906 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06421a61-5c52-4710-af42-0e9bb56d8f2a None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:54.689009 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06421a61-5c52-4710-af42-0e9bb56d8f2a None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:54.689009 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06421a61-5c52-4710-af42-0e9bb56d8f2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:54.728007 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:17:54.728513 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-06421a61-5c52-4710-af42-0e9bb56d8f2a None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:17:54.729777 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-06421a61-5c52-4710-af42-0e9bb56d8f2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:17:54.749780 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274d5864-5fca-4b34-b8d7-4c8c64bce262 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:17:54.751462 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274d5864-5fca-4b34-b8d7-4c8c64bce262 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:54.752346 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274d5864-5fca-4b34-b8d7-4c8c64bce262 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:54.752836 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274d5864-5fca-4b34-b8d7-4c8c64bce262 admin admin] PATH_INFO: `/v3/users/c3b90a10ca674c59a20171b30843a520` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:54.754452 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274d5864-5fca-4b34-b8d7-4c8c64bce262 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c3b90a10ca674c59a20171b30843a520)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:17:54.756862 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274d5864-5fca-4b34-b8d7-4c8c64bce262 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:17:54.821900 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1021/2050] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:17:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 06:17:54.823803 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a76a6b-d6c8-413c-9061-b6bb434d5389 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:54.824081 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a76a6b-d6c8-413c-9061-b6bb434d5389 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:54.824418 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a76a6b-d6c8-413c-9061-b6bb434d5389 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:54.846799 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-14a76a6b-d6c8-413c-9061-b6bb434d5389 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:17:54.848026 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-14a76a6b-d6c8-413c-9061-b6bb434d5389 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:17:54.859509 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1022/2051] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:17:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:17:54.861837 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a76a6b-d6c8-413c-9061-b6bb434d5389 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:17:54.871473 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a76a6b-d6c8-413c-9061-b6bb434d5389 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:17:54.876326 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88075822-b732-4189-a5d6-3c32c113b49a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:17:54.876975 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88075822-b732-4189-a5d6-3c32c113b49a admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:54.878743 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88075822-b732-4189-a5d6-3c32c113b49a admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:54.879029 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88075822-b732-4189-a5d6-3c32c113b49a admin admin] PATH_INFO: `/v3/domains` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:54.879690 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88075822-b732-4189-a5d6-3c32c113b49a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:17:54.879996 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88075822-b732-4189-a5d6-3c32c113b49a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:17:54.881645 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88075822-b732-4189-a5d6-3c32c113b49a admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:17:54.894606 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1023/2052] 10.222.0.62 () {58 vars in 1153 bytes} [Wed Aug 24 06:17:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:17:54.899110 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88075822-b732-4189-a5d6-3c32c113b49a admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:17:54.909551 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88075822-b732-4189-a5d6-3c32c113b49a admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:17:54.915858 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fb9aaf-3ab9-447e-856f-b260d9caa7e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:17:54.916476 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fb9aaf-3ab9-447e-856f-b260d9caa7e2 admin admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:54.916790 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fb9aaf-3ab9-447e-856f-b260d9caa7e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:54.917064 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fb9aaf-3ab9-447e-856f-b260d9caa7e2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:54.917893 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fb9aaf-3ab9-447e-856f-b260d9caa7e2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:17:54.919977 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fb9aaf-3ab9-447e-856f-b260d9caa7e2 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:17:55.026763 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-274d5864-5fca-4b34-b8d7-4c8c64bce262 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c3b90a10ca674c59a20171b30843a520'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:17:55.038149 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1024/2053] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Aug 24 06:17:54 2022] POST /identity/v3/projects => generated 375 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 06:17:55.042510 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fb9aaf-3ab9-447e-856f-b260d9caa7e2 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:17:55.046571 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-274d5864-5fca-4b34-b8d7-4c8c64bce262 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c3b90a10ca674c59a20171b30843a520'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:17:55.047278 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-274d5864-5fca-4b34-b8d7-4c8c64bce262 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c3b90a10ca674c59a20171b30843a520'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:17:55.047917 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-274d5864-5fca-4b34-b8d7-4c8c64bce262 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c3b90a10ca674c59a20171b30843a520'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:17:55.108358 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:17:55.113631 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fb9aaf-3ab9-447e-856f-b260d9caa7e2 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:17:55.119255 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4160bdd-e3c3-45ef-a020-1f617d3af8d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:17:55.119926 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4160bdd-e3c3-45ef-a020-1f617d3af8d7 admin admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:55.120198 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4160bdd-e3c3-45ef-a020-1f617d3af8d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:55.123973 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4160bdd-e3c3-45ef-a020-1f617d3af8d7 admin admin] PATH_INFO: `/v3/users` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:55.123973 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4160bdd-e3c3-45ef-a020-1f617d3af8d7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:17:55.123973 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4160bdd-e3c3-45ef-a020-1f617d3af8d7 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:17:55.142172 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-274d5864-5fca-4b34-b8d7-4c8c64bce262 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c3b90a10ca674c59a20171b30843a520'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:17:55.165946 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1030/2054] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Aug 24 06:17:54 2022] DELETE /identity/v3/users/c3b90a10ca674c59a20171b30843a520 => generated 0 bytes in 544 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 06:17:55.170187 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057cde46-5a33-4459-b063-be25a65782f4 None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:55.170476 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057cde46-5a33-4459-b063-be25a65782f4 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:55.170830 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057cde46-5a33-4459-b063-be25a65782f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:55.192427 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-057cde46-5a33-4459-b063-be25a65782f4 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:17:55.193906 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-057cde46-5a33-4459-b063-be25a65782f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:17:55.211859 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1025/2055] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Aug 24 06:17:55 2022] POST /identity/v3/users => generated 288 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:17:55.216092 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4160bdd-e3c3-45ef-a020-1f617d3af8d7 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:17:55.217830 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-d4160bdd-e3c3-45ef-a020-1f617d3af8d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:17:55.230482 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4160bdd-e3c3-45ef-a020-1f617d3af8d7 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:17:55.283311 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1031/2056] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:17:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 06:17:55.296665 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057cde46-5a33-4459-b063-be25a65782f4 None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:17:55.309641 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-057cde46-5a33-4459-b063-be25a65782f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:17:55.325116 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79002c43-d222-43f9-a390-9ddb7e0a39b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:17:55.325788 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79002c43-d222-43f9-a390-9ddb7e0a39b1 admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:55.334250 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79002c43-d222-43f9-a390-9ddb7e0a39b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:55.334793 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79002c43-d222-43f9-a390-9ddb7e0a39b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:55.335507 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79002c43-d222-43f9-a390-9ddb7e0a39b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:17:55.339062 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79002c43-d222-43f9-a390-9ddb7e0a39b1 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:17:55.353979 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1026/2057] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Aug 24 06:17:55 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 06:17:55.355516 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79002c43-d222-43f9-a390-9ddb7e0a39b1 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:17:55.379661 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79002c43-d222-43f9-a390-9ddb7e0a39b1 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:17:55.385584 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cc1b0f-7ce1-440b-b9a7-0cf3c3653099 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:17:55.387021 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cc1b0f-7ce1-440b-b9a7-0cf3c3653099 admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:55.387532 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cc1b0f-7ce1-440b-b9a7-0cf3c3653099 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:55.387963 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057cde46-5a33-4459-b063-be25a65782f4 None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:17:55.388251 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cc1b0f-7ce1-440b-b9a7-0cf3c3653099 admin admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:55.389325 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07cc1b0f-7ce1-440b-b9a7-0cf3c3653099 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:17:55.392101 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07cc1b0f-7ce1-440b-b9a7-0cf3c3653099 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:17:55.392829 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc206abc-da2a-4650-95ae-fa1c9e21a6e7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bb6c79db75e4d169331c02133d1ec32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:17:55.393737 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc206abc-da2a-4650-95ae-fa1c9e21a6e7 service neutron] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:55.394336 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc206abc-da2a-4650-95ae-fa1c9e21a6e7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:55.394889 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc206abc-da2a-4650-95ae-fa1c9e21a6e7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:55.407987 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc206abc-da2a-4650-95ae-fa1c9e21a6e7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:17:55.410385 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc206abc-da2a-4650-95ae-fa1c9e21a6e7 service neutron] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:17:55.419923 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1027/2058] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Aug 24 06:17:55 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:17:55.421091 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cc1b0f-7ce1-440b-b9a7-0cf3c3653099 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:17:55.432267 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cc1b0f-7ce1-440b-b9a7-0cf3c3653099 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:17:55.451632 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d61bb1-6957-4859-9022-b778df6a6d0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:17:55.452451 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d61bb1-6957-4859-9022-b778df6a6d0d admin admin] REQUEST_METHOD: `PUT` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:55.452817 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d61bb1-6957-4859-9022-b778df6a6d0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:55.453172 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d61bb1-6957-4859-9022-b778df6a6d0d admin admin] PATH_INFO: `/v3/projects/e2345e03d1a94b1ea2388246aeefedbb/users/c9007aaaf32d4c22b3dc56c3cd1e08ed/roles/580aa7dbb38d4cff8f9d1bb8983f99ef` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:55.454274 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1032/2059] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:17:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:17:55.488575 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d61bb1-6957-4859-9022-b778df6a6d0d admin admin] RBAC: Authorizing `identity:create_grant(project_id=e2345e03d1a94b1ea2388246aeefedbb, user_id=c9007aaaf32d4c22b3dc56c3cd1e08ed, role_id=580aa7dbb38d4cff8f9d1bb8983f99ef)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:17:55.491345 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d61bb1-6957-4859-9022-b778df6a6d0d admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:17:55.576941 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1028/2060] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Aug 24 06:17:55 2022] PUT /identity/v3/projects/e2345e03d1a94b1ea2388246aeefedbb/users/c9007aaaf32d4c22b3dc56c3cd1e08ed/roles/580aa7dbb38d4cff8f9d1bb8983f99ef => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:17:55.578901 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36115fd-6aa5-46c4-899d-0a28c6f3b883 None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:55.579195 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36115fd-6aa5-46c4-899d-0a28c6f3b883 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:55.579475 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36115fd-6aa5-46c4-899d-0a28c6f3b883 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:55.601379 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:17:55.609295 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-a36115fd-6aa5-46c4-899d-0a28c6f3b883 None None] MFA Rules not processed for user `c9007aaaf32d4c22b3dc56c3cd1e08ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:17:55.610646 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-a36115fd-6aa5-46c4-899d-0a28c6f3b883 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:17:55.687877 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1033/2061] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:17:55 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:17:55.698920 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d61bb1-6957-4859-9022-b778df6a6d0d admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:17:55.700830 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-97d61bb1-6957-4859-9022-b778df6a6d0d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:17:55.781792 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d61bb1-6957-4859-9022-b778df6a6d0d admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:17:55.786537 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917e1dfb-c55e-4de7-9d83-8e07baf811c0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c4def55ac0e48b29836bc5c64bc7402', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:17:55.787146 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917e1dfb-c55e-4de7-9d83-8e07baf811c0 service cinder] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:55.787439 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917e1dfb-c55e-4de7-9d83-8e07baf811c0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:55.787758 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917e1dfb-c55e-4de7-9d83-8e07baf811c0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:55.797072 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917e1dfb-c55e-4de7-9d83-8e07baf811c0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:17:55.798653 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917e1dfb-c55e-4de7-9d83-8e07baf811c0 service cinder] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:17:55.810186 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1029/2062] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:17:55 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:17:55.875901 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36115fd-6aa5-46c4-899d-0a28c6f3b883 None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:17:55.881705 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-a36115fd-6aa5-46c4-899d-0a28c6f3b883 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:17:55.984196 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36115fd-6aa5-46c4-899d-0a28c6f3b883 None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:17:55.990203 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8fe17c-f545-4d7b-97aa-5b6348f93d9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:17:55.990938 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8fe17c-f545-4d7b-97aa-5b6348f93d9d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:55.991220 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8fe17c-f545-4d7b-97aa-5b6348f93d9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:55.991490 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8fe17c-f545-4d7b-97aa-5b6348f93d9d admin admin] PATH_INFO: `/v3/projects/1239f0f5e26d4b0d81f4f57df1b4d578` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:55.992713 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8fe17c-f545-4d7b-97aa-5b6348f93d9d admin admin] RBAC: Authorizing `identity:delete_project(project_id=1239f0f5e26d4b0d81f4f57df1b4d578)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:17:55.994825 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8fe17c-f545-4d7b-97aa-5b6348f93d9d admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:17:56.137791 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-7c8fe17c-f545-4d7b-97aa-5b6348f93d9d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1239f0f5e26d4b0d81f4f57df1b4d578', 'request_id': 'req-7c8fe17c-f545-4d7b-97aa-5b6348f93d9d'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:17:56.150865 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-7c8fe17c-f545-4d7b-97aa-5b6348f93d9d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1239f0f5e26d4b0d81f4f57df1b4d578', 'request_id': 'req-7c8fe17c-f545-4d7b-97aa-5b6348f93d9d'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:17:56.158483 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-7c8fe17c-f545-4d7b-97aa-5b6348f93d9d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1239f0f5e26d4b0d81f4f57df1b4d578', 'request_id': 'req-7c8fe17c-f545-4d7b-97aa-5b6348f93d9d'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:17:56.286804 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-7c8fe17c-f545-4d7b-97aa-5b6348f93d9d admin admin] The token cache is being invalidate because project 1239f0f5e26d4b0d81f4f57df1b4d578 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81851) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:17:56.287094 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-7c8fe17c-f545-4d7b-97aa-5b6348f93d9d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:17:56.287748 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-7c8fe17c-f545-4d7b-97aa-5b6348f93d9d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:17:56.289650 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1034/2063] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Aug 24 06:17:55 2022] DELETE /identity/v3/projects/1239f0f5e26d4b0d81f4f57df1b4d578 => generated 0 bytes in 414 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:17:56.293755 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917e1dfb-c55e-4de7-9d83-8e07baf811c0 service cinder] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:17:56.308444 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917e1dfb-c55e-4de7-9d83-8e07baf811c0 service cinder] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:17:56.366898 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:17:56.380118 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76090ba6-4b34-4a97-9c69-a5651b7c9797 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:17:56.381903 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76090ba6-4b34-4a97-9c69-a5651b7c9797 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:56.382274 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76090ba6-4b34-4a97-9c69-a5651b7c9797 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:56.382960 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76090ba6-4b34-4a97-9c69-a5651b7c9797 admin admin] PATH_INFO: `/v3/users/af357606dd7c4ef39dcc1654755fdbb9` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:56.384068 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76090ba6-4b34-4a97-9c69-a5651b7c9797 admin admin] RBAC: Authorizing `identity:delete_user(user_id=af357606dd7c4ef39dcc1654755fdbb9)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:17:56.386154 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76090ba6-4b34-4a97-9c69-a5651b7c9797 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:17:56.580119 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-76090ba6-4b34-4a97-9c69-a5651b7c9797 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'af357606dd7c4ef39dcc1654755fdbb9'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:17:56.587725 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-76090ba6-4b34-4a97-9c69-a5651b7c9797 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'af357606dd7c4ef39dcc1654755fdbb9'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:17:56.588610 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-76090ba6-4b34-4a97-9c69-a5651b7c9797 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'af357606dd7c4ef39dcc1654755fdbb9'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:17:56.714212 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-76090ba6-4b34-4a97-9c69-a5651b7c9797 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'af357606dd7c4ef39dcc1654755fdbb9'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:17:56.732022 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-76090ba6-4b34-4a97-9c69-a5651b7c9797 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'af357606dd7c4ef39dcc1654755fdbb9'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:17:56.739576 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1030/2064] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Aug 24 06:17:56 2022] DELETE /identity/v3/users/af357606dd7c4ef39dcc1654755fdbb9 => generated 0 bytes in 446 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:17:57.195350 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8fe17c-f545-4d7b-97aa-5b6348f93d9d admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:17:57.200057 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-7c8fe17c-f545-4d7b-97aa-5b6348f93d9d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:17:57.212894 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8fe17c-f545-4d7b-97aa-5b6348f93d9d admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:17:57.247648 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:17:57.272921 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaaf276b-9227-429b-b8f3-0ba6b0351b06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:17:57.273844 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaaf276b-9227-429b-b8f3-0ba6b0351b06 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:57.274334 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaaf276b-9227-429b-b8f3-0ba6b0351b06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:57.274660 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaaf276b-9227-429b-b8f3-0ba6b0351b06 admin admin] PATH_INFO: `/v3/users/83b007978b5f48519f8ccec3844977e5` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:57.275983 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaaf276b-9227-429b-b8f3-0ba6b0351b06 admin admin] RBAC: Authorizing `identity:delete_user(user_id=83b007978b5f48519f8ccec3844977e5)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:17:57.278143 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaaf276b-9227-429b-b8f3-0ba6b0351b06 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:17:57.303729 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76090ba6-4b34-4a97-9c69-a5651b7c9797 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:17:57.305221 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-76090ba6-4b34-4a97-9c69-a5651b7c9797 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:17:57.319874 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76090ba6-4b34-4a97-9c69-a5651b7c9797 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:17:57.324635 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098b9ba6-b4f7-43a8-b95b-661cc1f1dcf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:17:57.325296 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098b9ba6-b4f7-43a8-b95b-661cc1f1dcf6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:57.325620 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098b9ba6-b4f7-43a8-b95b-661cc1f1dcf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:57.325919 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098b9ba6-b4f7-43a8-b95b-661cc1f1dcf6 admin admin] PATH_INFO: `/v3/projects/56b1757ce49c40afa993d692f279f729` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:57.327083 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098b9ba6-b4f7-43a8-b95b-661cc1f1dcf6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=56b1757ce49c40afa993d692f279f729)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:17:57.331556 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098b9ba6-b4f7-43a8-b95b-661cc1f1dcf6 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:17:58.189862 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-aaaf276b-9227-429b-b8f3-0ba6b0351b06 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '83b007978b5f48519f8ccec3844977e5'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:17:58.192529 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-098b9ba6-b4f7-43a8-b95b-661cc1f1dcf6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '56b1757ce49c40afa993d692f279f729', 'request_id': 'req-098b9ba6-b4f7-43a8-b95b-661cc1f1dcf6'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:17:58.197539 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-aaaf276b-9227-429b-b8f3-0ba6b0351b06 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '83b007978b5f48519f8ccec3844977e5'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:17:58.198339 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-098b9ba6-b4f7-43a8-b95b-661cc1f1dcf6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '56b1757ce49c40afa993d692f279f729', 'request_id': 'req-098b9ba6-b4f7-43a8-b95b-661cc1f1dcf6'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:17:58.198587 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-aaaf276b-9227-429b-b8f3-0ba6b0351b06 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '83b007978b5f48519f8ccec3844977e5'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:17:58.198932 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-aaaf276b-9227-429b-b8f3-0ba6b0351b06 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '83b007978b5f48519f8ccec3844977e5'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:17:58.205007 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-098b9ba6-b4f7-43a8-b95b-661cc1f1dcf6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '56b1757ce49c40afa993d692f279f729', 'request_id': 'req-098b9ba6-b4f7-43a8-b95b-661cc1f1dcf6'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:17:58.262790 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-aaaf276b-9227-429b-b8f3-0ba6b0351b06 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '83b007978b5f48519f8ccec3844977e5'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:17:58.280531 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1035/2065] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Aug 24 06:17:57 2022] DELETE /identity/v3/users/83b007978b5f48519f8ccec3844977e5 => generated 0 bytes in 1086 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:17:58.281757 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84bbb95-5604-4bff-b4a7-bd96fe3d3576 None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:58.282048 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84bbb95-5604-4bff-b4a7-bd96fe3d3576 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:58.282316 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84bbb95-5604-4bff-b4a7-bd96fe3d3576 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:58.301103 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-b84bbb95-5604-4bff-b4a7-bd96fe3d3576 None None] MFA Rules not processed for user `c9007aaaf32d4c22b3dc56c3cd1e08ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:17:58.302287 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-b84bbb95-5604-4bff-b4a7-bd96fe3d3576 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:17:58.322822 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-098b9ba6-b4f7-43a8-b95b-661cc1f1dcf6 admin admin] The token cache is being invalidate because project 56b1757ce49c40afa993d692f279f729 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81852) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:17:58.322822 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-098b9ba6-b4f7-43a8-b95b-661cc1f1dcf6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:17:58.323290 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-098b9ba6-b4f7-43a8-b95b-661cc1f1dcf6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:17:58.325617 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1031/2066] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Aug 24 06:17:57 2022] DELETE /identity/v3/projects/56b1757ce49c40afa993d692f279f729 => generated 0 bytes in 1022 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 06:17:58.332536 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add8b101-62ef-4881-b867-0ee0534c23b5 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:58.332536 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add8b101-62ef-4881-b867-0ee0534c23b5 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:58.332536 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add8b101-62ef-4881-b867-0ee0534c23b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:58.359751 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-add8b101-62ef-4881-b867-0ee0534c23b5 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:17:58.361688 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-add8b101-62ef-4881-b867-0ee0534c23b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:17:58.369966 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1036/2067] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:17:58 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:17:58.369966 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7fbb82-af4d-45fe-86b3-7e5bd7128fb3 None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:58.369966 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7fbb82-af4d-45fe-86b3-7e5bd7128fb3 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:58.369966 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7fbb82-af4d-45fe-86b3-7e5bd7128fb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:58.411480 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-cd7fbb82-af4d-45fe-86b3-7e5bd7128fb3 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:17:58.412750 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-cd7fbb82-af4d-45fe-86b3-7e5bd7128fb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:17:58.416540 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:17:58.440481 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1032/2068] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:17:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:17:58.454798 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add8b101-62ef-4881-b867-0ee0534c23b5 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:17:58.457247 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-add8b101-62ef-4881-b867-0ee0534c23b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:17:58.562041 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1037/2069] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:17:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:17:58.564456 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dee690-25f6-4f5a-931a-e023b2ea0921 None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:58.564995 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dee690-25f6-4f5a-931a-e023b2ea0921 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:58.565483 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dee690-25f6-4f5a-931a-e023b2ea0921 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:58.565792 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add8b101-62ef-4881-b867-0ee0534c23b5 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:17:58.575183 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd08ee1-1991-43eb-8b73-e541e5f9a366 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bb6c79db75e4d169331c02133d1ec32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:17:58.593582 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd08ee1-1991-43eb-8b73-e541e5f9a366 service neutron] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:58.593582 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd08ee1-1991-43eb-8b73-e541e5f9a366 service neutron] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:58.593582 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd08ee1-1991-43eb-8b73-e541e5f9a366 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:58.593582 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd08ee1-1991-43eb-8b73-e541e5f9a366 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:17:58.593582 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd08ee1-1991-43eb-8b73-e541e5f9a366 service neutron] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:17:58.623753 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-31dee690-25f6-4f5a-931a-e023b2ea0921 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:17:58.625112 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-31dee690-25f6-4f5a-931a-e023b2ea0921 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:17:58.627322 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1033/2070] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:17:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 06:17:58.635456 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1038/2071] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:17:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:17:58.640076 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd08ee1-1991-43eb-8b73-e541e5f9a366 service neutron] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:17:58.646540 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:17:58.652042 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd08ee1-1991-43eb-8b73-e541e5f9a366 service neutron] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:17:58.668340 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3f86da-72e3-4901-858a-ca882810b1dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:17:58.669519 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3f86da-72e3-4901-858a-ca882810b1dd admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:58.670333 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3f86da-72e3-4901-858a-ca882810b1dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:58.670821 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3f86da-72e3-4901-858a-ca882810b1dd admin admin] PATH_INFO: `/v3/domains` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:58.671725 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3f86da-72e3-4901-858a-ca882810b1dd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:17:58.672360 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3f86da-72e3-4901-858a-ca882810b1dd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:17:58.675104 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3f86da-72e3-4901-858a-ca882810b1dd admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:17:58.717224 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1034/2072] 10.222.0.62 () {58 vars in 1153 bytes} [Wed Aug 24 06:17:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:17:58.719885 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31dee690-25f6-4f5a-931a-e023b2ea0921 None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:17:58.730380 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31dee690-25f6-4f5a-931a-e023b2ea0921 None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:17:58.743518 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fed3d9-5d30-4259-8fa4-18cb11ec2776 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:17:58.747998 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fed3d9-5d30-4259-8fa4-18cb11ec2776 admin admin] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:58.752119 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fed3d9-5d30-4259-8fa4-18cb11ec2776 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:58.754417 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fed3d9-5d30-4259-8fa4-18cb11ec2776 admin admin] PATH_INFO: `/v3/projects` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:58.754417 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fed3d9-5d30-4259-8fa4-18cb11ec2776 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:17:58.755319 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fed3d9-5d30-4259-8fa4-18cb11ec2776 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:17:58.883681 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1039/2073] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Aug 24 06:17:58 2022] POST /identity/v3/projects => generated 373 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:17:58.887901 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3f86da-72e3-4901-858a-ca882810b1dd admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:17:58.899662 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3f86da-72e3-4901-858a-ca882810b1dd admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:17:59.032226 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93db8cc-226a-47d5-a823-e83759f8a9b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:17:59.037214 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93db8cc-226a-47d5-a823-e83759f8a9b1 admin admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:59.037540 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93db8cc-226a-47d5-a823-e83759f8a9b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:59.037906 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93db8cc-226a-47d5-a823-e83759f8a9b1 admin admin] PATH_INFO: `/v3/users` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:59.038795 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93db8cc-226a-47d5-a823-e83759f8a9b1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:17:59.044687 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93db8cc-226a-47d5-a823-e83759f8a9b1 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:17:59.196705 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1035/2074] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Aug 24 06:17:58 2022] POST /identity/v3/users => generated 287 bytes in 310 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 06:17:59.200888 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fed3d9-5d30-4259-8fa4-18cb11ec2776 admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:17:59.212138 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fed3d9-5d30-4259-8fa4-18cb11ec2776 admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:17:59.217257 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68eabfe0-c349-4843-a90a-c92d33a02baa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:17:59.217914 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68eabfe0-c349-4843-a90a-c92d33a02baa admin admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:59.218176 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68eabfe0-c349-4843-a90a-c92d33a02baa admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:59.218506 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68eabfe0-c349-4843-a90a-c92d33a02baa admin admin] PATH_INFO: `/v3/roles` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:59.219759 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68eabfe0-c349-4843-a90a-c92d33a02baa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:17:59.221728 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68eabfe0-c349-4843-a90a-c92d33a02baa admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:17:59.232680 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1040/2075] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Aug 24 06:17:59 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:17:59.236862 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93db8cc-226a-47d5-a823-e83759f8a9b1 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:17:59.247654 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93db8cc-226a-47d5-a823-e83759f8a9b1 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:17:59.253848 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7736da44-3443-4adb-8ac0-bafb5bc14a22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:17:59.254590 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7736da44-3443-4adb-8ac0-bafb5bc14a22 admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:59.264891 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7736da44-3443-4adb-8ac0-bafb5bc14a22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:59.264891 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7736da44-3443-4adb-8ac0-bafb5bc14a22 admin admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:59.264891 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7736da44-3443-4adb-8ac0-bafb5bc14a22 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:17:59.264891 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7736da44-3443-4adb-8ac0-bafb5bc14a22 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:17:59.271822 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1036/2076] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Aug 24 06:17:59 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:17:59.281242 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68eabfe0-c349-4843-a90a-c92d33a02baa admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:17:59.292977 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7736da44-3443-4adb-8ac0-bafb5bc14a22 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:17:59.297819 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-7736da44-3443-4adb-8ac0-bafb5bc14a22 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:17:59.307792 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68eabfe0-c349-4843-a90a-c92d33a02baa admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:17:59.322317 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6144110f-d51f-47fe-92d1-d1bc069a1a70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:17:59.323308 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6144110f-d51f-47fe-92d1-d1bc069a1a70 admin admin] REQUEST_METHOD: `PUT` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:59.323983 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6144110f-d51f-47fe-92d1-d1bc069a1a70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:59.324414 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6144110f-d51f-47fe-92d1-d1bc069a1a70 admin admin] PATH_INFO: `/v3/projects/d80f5c74d8f74bc4bf780c1e3be66b05/users/7dc34e089e7e4475b006436dfa4effff/roles/580aa7dbb38d4cff8f9d1bb8983f99ef` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:59.329062 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7736da44-3443-4adb-8ac0-bafb5bc14a22 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:17:59.354142 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feddf7f5-bbaa-4c41-87aa-63983de7fbec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:17:59.357587 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feddf7f5-bbaa-4c41-87aa-63983de7fbec admin admin] REQUEST_METHOD: `DELETE` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:59.357587 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feddf7f5-bbaa-4c41-87aa-63983de7fbec admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:59.357587 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feddf7f5-bbaa-4c41-87aa-63983de7fbec admin admin] PATH_INFO: `/v3/projects/84a0d30512134d2ab6fcb752de4c51a2` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:59.358055 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feddf7f5-bbaa-4c41-87aa-63983de7fbec admin admin] RBAC: Authorizing `identity:delete_project(project_id=84a0d30512134d2ab6fcb752de4c51a2)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:17:59.360095 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feddf7f5-bbaa-4c41-87aa-63983de7fbec admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:17:59.369590 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6144110f-d51f-47fe-92d1-d1bc069a1a70 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d80f5c74d8f74bc4bf780c1e3be66b05, user_id=7dc34e089e7e4475b006436dfa4effff, role_id=580aa7dbb38d4cff8f9d1bb8983f99ef)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:17:59.379751 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6144110f-d51f-47fe-92d1-d1bc069a1a70 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:17:59.447393 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1041/2077] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Aug 24 06:17:59 2022] PUT /identity/v3/projects/d80f5c74d8f74bc4bf780c1e3be66b05/users/7dc34e089e7e4475b006436dfa4effff/roles/580aa7dbb38d4cff8f9d1bb8983f99ef => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:17:59.460541 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cc20d4-a71b-41f1-80b8-302007fd76ba None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:59.461214 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cc20d4-a71b-41f1-80b8-302007fd76ba None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:59.461712 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cc20d4-a71b-41f1-80b8-302007fd76ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:59.515505 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-feddf7f5-bbaa-4c41-87aa-63983de7fbec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '84a0d30512134d2ab6fcb752de4c51a2', 'request_id': 'req-feddf7f5-bbaa-4c41-87aa-63983de7fbec'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:17:59.524612 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-d2cc20d4-a71b-41f1-80b8-302007fd76ba None None] MFA Rules not processed for user `7dc34e089e7e4475b006436dfa4effff`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:17:59.524772 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=91,keystone:INSERT=3,keystone:DELETE=6 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:17:59.526214 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-feddf7f5-bbaa-4c41-87aa-63983de7fbec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '84a0d30512134d2ab6fcb752de4c51a2', 'request_id': 'req-feddf7f5-bbaa-4c41-87aa-63983de7fbec'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:17:59.530166 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-d2cc20d4-a71b-41f1-80b8-302007fd76ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:17:59.536592 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-feddf7f5-bbaa-4c41-87aa-63983de7fbec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '84a0d30512134d2ab6fcb752de4c51a2', 'request_id': 'req-feddf7f5-bbaa-4c41-87aa-63983de7fbec'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:17:59.629680 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-feddf7f5-bbaa-4c41-87aa-63983de7fbec admin admin] The token cache is being invalidate because project 84a0d30512134d2ab6fcb752de4c51a2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81852) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:17:59.629680 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-feddf7f5-bbaa-4c41-87aa-63983de7fbec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:17:59.629680 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-feddf7f5-bbaa-4c41-87aa-63983de7fbec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:17:59.629680 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1037/2078] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Aug 24 06:17:59 2022] DELETE /identity/v3/projects/84a0d30512134d2ab6fcb752de4c51a2 => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:17:59.632831 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feddf7f5-bbaa-4c41-87aa-63983de7fbec admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:17:59.635865 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1042/2079] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:17:59 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:17:59.636735 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-feddf7f5-bbaa-4c41-87aa-63983de7fbec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:17:59.650002 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cc20d4-a71b-41f1-80b8-302007fd76ba None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:17:59.652058 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-d2cc20d4-a71b-41f1-80b8-302007fd76ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:17:59.782317 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cc20d4-a71b-41f1-80b8-302007fd76ba None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:17:59.786300 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea0b67c-482c-4d16-b766-558c412f2694 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c4def55ac0e48b29836bc5c64bc7402', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:17:59.786935 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea0b67c-482c-4d16-b766-558c412f2694 service cinder] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:59.787343 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea0b67c-482c-4d16-b766-558c412f2694 service cinder] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:59.787519 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea0b67c-482c-4d16-b766-558c412f2694 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:59.789553 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-cea0b67c-482c-4d16-b766-558c412f2694 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:17:59.807128 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea0b67c-482c-4d16-b766-558c412f2694 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:17:59.810239 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea0b67c-482c-4d16-b766-558c412f2694 service cinder] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:17:59.820940 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feddf7f5-bbaa-4c41-87aa-63983de7fbec admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:17:59.822131 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1043/2080] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:17:59 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 06:17:59.826162 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538e475c-3675-42fb-9e82-df420140f7f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:17:59.826801 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538e475c-3675-42fb-9e82-df420140f7f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:59.827086 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538e475c-3675-42fb-9e82-df420140f7f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:59.827364 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538e475c-3675-42fb-9e82-df420140f7f6 admin admin] PATH_INFO: `/v3/users/ba8f435de9d749aa9243f7aaba13d192` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:59.828525 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538e475c-3675-42fb-9e82-df420140f7f6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ba8f435de9d749aa9243f7aaba13d192)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:17:59.830530 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538e475c-3675-42fb-9e82-df420140f7f6 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:17:59.843933 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea0b67c-482c-4d16-b766-558c412f2694 service cinder] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:17:59.846113 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-cea0b67c-482c-4d16-b766-558c412f2694 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:17:59.865518 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea0b67c-482c-4d16-b766-558c412f2694 service cinder] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:17:59.870374 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed05067-2d2f-41bb-a994-a40dd67333c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:17:59.871110 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed05067-2d2f-41bb-a994-a40dd67333c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:17:59.871604 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed05067-2d2f-41bb-a994-a40dd67333c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:17:59.871939 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed05067-2d2f-41bb-a994-a40dd67333c6 admin admin] PATH_INFO: `/v3/users/7dc34e089e7e4475b006436dfa4effff` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:17:59.873180 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed05067-2d2f-41bb-a994-a40dd67333c6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7dc34e089e7e4475b006436dfa4effff)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:17:59.875174 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed05067-2d2f-41bb-a994-a40dd67333c6 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:17:59.900165 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=96,keystone:INSERT=2,keystone:DELETE=2 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:18:00.109758 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-538e475c-3675-42fb-9e82-df420140f7f6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ba8f435de9d749aa9243f7aaba13d192'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:18:00.114367 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-6ed05067-2d2f-41bb-a994-a40dd67333c6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7dc34e089e7e4475b006436dfa4effff'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:18:00.118370 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-538e475c-3675-42fb-9e82-df420140f7f6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ba8f435de9d749aa9243f7aaba13d192'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:18:00.119098 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-538e475c-3675-42fb-9e82-df420140f7f6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ba8f435de9d749aa9243f7aaba13d192'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:18:00.122082 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-6ed05067-2d2f-41bb-a994-a40dd67333c6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7dc34e089e7e4475b006436dfa4effff'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:18:00.122762 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-6ed05067-2d2f-41bb-a994-a40dd67333c6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7dc34e089e7e4475b006436dfa4effff'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:18:00.123682 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-6ed05067-2d2f-41bb-a994-a40dd67333c6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7dc34e089e7e4475b006436dfa4effff'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:18:00.239466 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-6ed05067-2d2f-41bb-a994-a40dd67333c6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7dc34e089e7e4475b006436dfa4effff'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:18:00.259377 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1044/2081] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Aug 24 06:17:59 2022] DELETE /identity/v3/users/7dc34e089e7e4475b006436dfa4effff => generated 0 bytes in 416 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:18:00.267920 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116dfd4b-64a4-4437-a6f6-16e9aa4199d7 None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:18:00.267920 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116dfd4b-64a4-4437-a6f6-16e9aa4199d7 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:18:00.267920 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116dfd4b-64a4-4437-a6f6-16e9aa4199d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:18:00.287350 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-116dfd4b-64a4-4437-a6f6-16e9aa4199d7 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:18:00.290035 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-116dfd4b-64a4-4437-a6f6-16e9aa4199d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:18:00.324858 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-538e475c-3675-42fb-9e82-df420140f7f6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ba8f435de9d749aa9243f7aaba13d192'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:18:00.346921 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-538e475c-3675-42fb-9e82-df420140f7f6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ba8f435de9d749aa9243f7aaba13d192'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:18:00.355315 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1038/2082] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Aug 24 06:17:59 2022] DELETE /identity/v3/users/ba8f435de9d749aa9243f7aaba13d192 => generated 0 bytes in 723 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:18:00.400534 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1045/2083] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:18:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 06:18:00.408843 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538e475c-3675-42fb-9e82-df420140f7f6 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:18:00.410875 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-538e475c-3675-42fb-9e82-df420140f7f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:18:00.478160 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538e475c-3675-42fb-9e82-df420140f7f6 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:18:00.482346 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983cd86c-4699-4f8c-99f0-474795b791cb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bb6c79db75e4d169331c02133d1ec32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:18:00.482948 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983cd86c-4699-4f8c-99f0-474795b791cb service neutron] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:18:00.483230 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983cd86c-4699-4f8c-99f0-474795b791cb service neutron] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:18:00.483505 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983cd86c-4699-4f8c-99f0-474795b791cb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:18:00.501896 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:18:00.565890 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983cd86c-4699-4f8c-99f0-474795b791cb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:18:00.568019 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983cd86c-4699-4f8c-99f0-474795b791cb service neutron] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:18:00.580365 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1039/2084] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:18:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:18:01.167673 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116dfd4b-64a4-4437-a6f6-16e9aa4199d7 None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:18:01.167923 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983cd86c-4699-4f8c-99f0-474795b791cb service neutron] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:18:01.169299 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-116dfd4b-64a4-4437-a6f6-16e9aa4199d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:18:01.171133 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-983cd86c-4699-4f8c-99f0-474795b791cb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:18:01.195178 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983cd86c-4699-4f8c-99f0-474795b791cb service neutron] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:18:01.198943 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116dfd4b-64a4-4437-a6f6-16e9aa4199d7 None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:18:01.205305 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d614a08a-3112-4a39-bd35-994b6267f3d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:18:01.206308 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8173e7e9-ba9c-4d9e-a1bb-383c28bac25b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:18:01.206956 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8173e7e9-ba9c-4d9e-a1bb-383c28bac25b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:18:01.207233 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8173e7e9-ba9c-4d9e-a1bb-383c28bac25b admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:18:01.207507 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8173e7e9-ba9c-4d9e-a1bb-383c28bac25b admin admin] PATH_INFO: `/v3/projects/d80f5c74d8f74bc4bf780c1e3be66b05` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:18:01.208407 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d614a08a-3112-4a39-bd35-994b6267f3d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:18:01.208698 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d614a08a-3112-4a39-bd35-994b6267f3d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:18:01.208968 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d614a08a-3112-4a39-bd35-994b6267f3d0 admin admin] PATH_INFO: `/v3/projects/5562f66712d34de5ba7e35b6d2955276` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:18:01.210103 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8173e7e9-ba9c-4d9e-a1bb-383c28bac25b admin admin] RBAC: Authorizing `identity:delete_project(project_id=d80f5c74d8f74bc4bf780c1e3be66b05)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:18:01.212026 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8173e7e9-ba9c-4d9e-a1bb-383c28bac25b admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:18:01.214150 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d614a08a-3112-4a39-bd35-994b6267f3d0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5562f66712d34de5ba7e35b6d2955276)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:18:01.216187 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d614a08a-3112-4a39-bd35-994b6267f3d0 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:18:01.349734 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-d614a08a-3112-4a39-bd35-994b6267f3d0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5562f66712d34de5ba7e35b6d2955276', 'request_id': 'req-d614a08a-3112-4a39-bd35-994b6267f3d0'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:18:01.364468 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-d614a08a-3112-4a39-bd35-994b6267f3d0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5562f66712d34de5ba7e35b6d2955276', 'request_id': 'req-d614a08a-3112-4a39-bd35-994b6267f3d0'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:18:01.370440 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-8173e7e9-ba9c-4d9e-a1bb-383c28bac25b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd80f5c74d8f74bc4bf780c1e3be66b05', 'request_id': 'req-8173e7e9-ba9c-4d9e-a1bb-383c28bac25b'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:18:01.381484 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-d614a08a-3112-4a39-bd35-994b6267f3d0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5562f66712d34de5ba7e35b6d2955276', 'request_id': 'req-d614a08a-3112-4a39-bd35-994b6267f3d0'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:18:01.383217 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-8173e7e9-ba9c-4d9e-a1bb-383c28bac25b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd80f5c74d8f74bc4bf780c1e3be66b05', 'request_id': 'req-8173e7e9-ba9c-4d9e-a1bb-383c28bac25b'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:18:01.395050 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-8173e7e9-ba9c-4d9e-a1bb-383c28bac25b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd80f5c74d8f74bc4bf780c1e3be66b05', 'request_id': 'req-8173e7e9-ba9c-4d9e-a1bb-383c28bac25b'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:18:01.491878 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-d614a08a-3112-4a39-bd35-994b6267f3d0 admin admin] The token cache is being invalidate because project 5562f66712d34de5ba7e35b6d2955276 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81852) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:18:01.492233 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-d614a08a-3112-4a39-bd35-994b6267f3d0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:18:01.492972 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-d614a08a-3112-4a39-bd35-994b6267f3d0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:18:01.494968 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1040/2085] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Aug 24 06:18:01 2022] DELETE /identity/v3/projects/5562f66712d34de5ba7e35b6d2955276 => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:18:01.500254 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adce280-d6b4-4da3-85c1-7217cff9c23e None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:18:01.500855 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adce280-d6b4-4da3-85c1-7217cff9c23e None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:18:01.501376 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adce280-d6b4-4da3-85c1-7217cff9c23e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:18:01.529073 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-0adce280-d6b4-4da3-85c1-7217cff9c23e None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:18:01.533883 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-0adce280-d6b4-4da3-85c1-7217cff9c23e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:18:01.558575 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-8173e7e9-ba9c-4d9e-a1bb-383c28bac25b admin admin] The token cache is being invalidate because project d80f5c74d8f74bc4bf780c1e3be66b05 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81851) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:18:01.559138 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-8173e7e9-ba9c-4d9e-a1bb-383c28bac25b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:18:01.560282 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-8173e7e9-ba9c-4d9e-a1bb-383c28bac25b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:18:01.563209 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1046/2086] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Aug 24 06:18:01 2022] DELETE /identity/v3/projects/d80f5c74d8f74bc4bf780c1e3be66b05 => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:18:01.568453 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07ae345-1871-438c-b9ae-c628533aecac None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:18:01.568893 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07ae345-1871-438c-b9ae-c628533aecac None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:18:01.569330 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07ae345-1871-438c-b9ae-c628533aecac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:18:01.611220 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-f07ae345-1871-438c-b9ae-c628533aecac None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:18:01.612440 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-f07ae345-1871-438c-b9ae-c628533aecac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:18:01.628823 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:DELETE=15,keystone:SELECT=83,keystone:INSERT=2 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:18:01.683951 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1041/2087] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:18:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:18:01.686104 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647c3583-75ff-45f9-a4a3-4caa373e97d3 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:18:01.686378 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647c3583-75ff-45f9-a4a3-4caa373e97d3 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:18:01.686684 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647c3583-75ff-45f9-a4a3-4caa373e97d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:18:01.715939 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1047/2088] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:18:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 06:18:01.717277 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-647c3583-75ff-45f9-a4a3-4caa373e97d3 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:18:01.718566 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-647c3583-75ff-45f9-a4a3-4caa373e97d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:18:01.724525 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5ccca2-aaa3-45d1-8859-7158a197c74d None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:18:01.724828 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5ccca2-aaa3-45d1-8859-7158a197c74d None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:18:01.725515 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5ccca2-aaa3-45d1-8859-7158a197c74d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:18:01.728533 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1042/2089] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:18:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:18:01.737078 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647c3583-75ff-45f9-a4a3-4caa373e97d3 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:18:01.747725 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647c3583-75ff-45f9-a4a3-4caa373e97d3 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:18:01.752360 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a4414e-a737-40b2-8673-e8cca42328f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:18:01.753587 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a4414e-a737-40b2-8673-e8cca42328f4 admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:18:01.753869 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a4414e-a737-40b2-8673-e8cca42328f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:18:01.754167 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a4414e-a737-40b2-8673-e8cca42328f4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:18:01.754856 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a4414e-a737-40b2-8673-e8cca42328f4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:18:01.755140 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a4414e-a737-40b2-8673-e8cca42328f4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:18:01.756760 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a4414e-a737-40b2-8673-e8cca42328f4 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:18:01.773742 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-0a5ccca2-aaa3-45d1-8859-7158a197c74d None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:18:01.775052 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-0a5ccca2-aaa3-45d1-8859-7158a197c74d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:18:01.776119 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1043/2090] 10.222.0.62 () {58 vars in 1153 bytes} [Wed Aug 24 06:18:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 06:18:01.776943 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a4414e-a737-40b2-8673-e8cca42328f4 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:18:01.783418 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1048/2091] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:18:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:18:01.787102 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a4414e-a737-40b2-8673-e8cca42328f4 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:18:01.787492 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5ccca2-aaa3-45d1-8859-7158a197c74d None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:18:01.791626 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebef4b94-88a7-4b30-9fc7-9c427dcc6b97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:18:01.792227 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebef4b94-88a7-4b30-9fc7-9c427dcc6b97 admin admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:18:01.792523 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebef4b94-88a7-4b30-9fc7-9c427dcc6b97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:18:01.792811 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebef4b94-88a7-4b30-9fc7-9c427dcc6b97 admin admin] PATH_INFO: `/v3/projects` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:18:01.793424 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebef4b94-88a7-4b30-9fc7-9c427dcc6b97 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:18:01.795328 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebef4b94-88a7-4b30-9fc7-9c427dcc6b97 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:18:01.797856 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5ccca2-aaa3-45d1-8859-7158a197c74d None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:18:01.802721 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fc1314-1f8f-4b78-84fe-4104d08b84b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:18:01.803324 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fc1314-1f8f-4b78-84fe-4104d08b84b7 admin admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:18:01.805600 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fc1314-1f8f-4b78-84fe-4104d08b84b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:18:01.807918 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fc1314-1f8f-4b78-84fe-4104d08b84b7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:18:01.809391 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fc1314-1f8f-4b78-84fe-4104d08b84b7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:18:01.809729 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fc1314-1f8f-4b78-84fe-4104d08b84b7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:18:01.812531 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fc1314-1f8f-4b78-84fe-4104d08b84b7 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:18:01.818137 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:18:01.824874 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1049/2092] 10.222.0.62 () {58 vars in 1153 bytes} [Wed Aug 24 06:18:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 06:18:01.829175 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fc1314-1f8f-4b78-84fe-4104d08b84b7 admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:18:01.839340 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fc1314-1f8f-4b78-84fe-4104d08b84b7 admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:18:01.844075 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cabec1-14a7-4aab-9d82-42fe1fbd30a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:18:01.844745 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cabec1-14a7-4aab-9d82-42fe1fbd30a9 admin admin] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:18:01.845027 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cabec1-14a7-4aab-9d82-42fe1fbd30a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:18:01.845304 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cabec1-14a7-4aab-9d82-42fe1fbd30a9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:18:01.846097 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7cabec1-14a7-4aab-9d82-42fe1fbd30a9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:18:01.848055 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7cabec1-14a7-4aab-9d82-42fe1fbd30a9 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:18:01.889172 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1044/2093] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Aug 24 06:18:01 2022] POST /identity/v3/projects => generated 365 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:18:01.900310 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebef4b94-88a7-4b30-9fc7-9c427dcc6b97 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:18:01.936544 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1050/2094] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Aug 24 06:18:01 2022] POST /identity/v3/projects => generated 369 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 06:18:01.938517 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cabec1-14a7-4aab-9d82-42fe1fbd30a9 admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:18:02.004557 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebef4b94-88a7-4b30-9fc7-9c427dcc6b97 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:18:02.036776 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cabec1-14a7-4aab-9d82-42fe1fbd30a9 admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:18:02.058133 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37651349-5c3e-4280-856c-5a4aa593b1ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:18:02.059380 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37651349-5c3e-4280-856c-5a4aa593b1ad admin admin] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:18:02.059715 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37651349-5c3e-4280-856c-5a4aa593b1ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:18:02.060044 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37651349-5c3e-4280-856c-5a4aa593b1ad admin admin] PATH_INFO: `/v3/users` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:18:02.063803 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37651349-5c3e-4280-856c-5a4aa593b1ad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:18:02.065864 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37651349-5c3e-4280-856c-5a4aa593b1ad admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:18:02.107444 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ec9eb8-7219-47fe-876d-101713075d3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:18:02.108288 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ec9eb8-7219-47fe-876d-101713075d3d admin admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:18:02.108654 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ec9eb8-7219-47fe-876d-101713075d3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:18:02.109128 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ec9eb8-7219-47fe-876d-101713075d3d admin admin] PATH_INFO: `/v3/users` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:18:02.109895 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ec9eb8-7219-47fe-876d-101713075d3d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:18:02.112097 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ec9eb8-7219-47fe-876d-101713075d3d admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:18:02.161221 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1051/2095] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Aug 24 06:18:01 2022] POST /identity/v3/users => generated 285 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:18:02.162488 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37651349-5c3e-4280-856c-5a4aa593b1ad admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:18:02.190621 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37651349-5c3e-4280-856c-5a4aa593b1ad admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:18:02.195794 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2c8163-1a11-4f47-ba67-9ae343b798e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:18:02.196569 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2c8163-1a11-4f47-ba67-9ae343b798e3 admin admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:18:02.197061 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2c8163-1a11-4f47-ba67-9ae343b798e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:18:02.197287 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2c8163-1a11-4f47-ba67-9ae343b798e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:18:02.197963 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2c8163-1a11-4f47-ba67-9ae343b798e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:18:02.199599 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2c8163-1a11-4f47-ba67-9ae343b798e3 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:18:02.200910 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1045/2096] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Aug 24 06:18:01 2022] POST /identity/v3/users => generated 283 bytes in 301 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:18:02.205589 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ec9eb8-7219-47fe-876d-101713075d3d admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:18:02.217547 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1052/2097] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Aug 24 06:18:02 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:18:02.217750 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ec9eb8-7219-47fe-876d-101713075d3d admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:18:02.221277 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2c8163-1a11-4f47-ba67-9ae343b798e3 admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:18:02.221412 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c68869-16dd-416a-b208-982be6bdba77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:18:02.221519 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c68869-16dd-416a-b208-982be6bdba77 admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:18:02.221815 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c68869-16dd-416a-b208-982be6bdba77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:18:02.222075 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c68869-16dd-416a-b208-982be6bdba77 admin admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:18:02.222722 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c68869-16dd-416a-b208-982be6bdba77 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:18:02.230514 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c68869-16dd-416a-b208-982be6bdba77 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:18:02.232358 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2c8163-1a11-4f47-ba67-9ae343b798e3 admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:18:02.239604 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1046/2098] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Aug 24 06:18:02 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 06:18:02.241406 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c68869-16dd-416a-b208-982be6bdba77 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:18:02.242135 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56dce3db-39f7-4054-89fa-2ca46f6dcef6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:18:02.242757 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dce3db-39f7-4054-89fa-2ca46f6dcef6 admin admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:18:02.243034 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dce3db-39f7-4054-89fa-2ca46f6dcef6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:18:02.243303 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dce3db-39f7-4054-89fa-2ca46f6dcef6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:18:02.243959 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56dce3db-39f7-4054-89fa-2ca46f6dcef6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:18:02.245707 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56dce3db-39f7-4054-89fa-2ca46f6dcef6 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:18:02.257746 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c68869-16dd-416a-b208-982be6bdba77 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:18:02.258940 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1053/2099] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Aug 24 06:18:02 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:18:02.263063 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56dce3db-39f7-4054-89fa-2ca46f6dcef6 admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:18:02.263610 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf43944-6d94-49dd-b018-9d8efbf6a216 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:18:02.264248 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf43944-6d94-49dd-b018-9d8efbf6a216 admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:18:02.264557 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf43944-6d94-49dd-b018-9d8efbf6a216 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:18:02.264836 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf43944-6d94-49dd-b018-9d8efbf6a216 admin admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:18:02.265557 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf43944-6d94-49dd-b018-9d8efbf6a216 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:18:02.267630 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf43944-6d94-49dd-b018-9d8efbf6a216 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:18:02.274940 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56dce3db-39f7-4054-89fa-2ca46f6dcef6 admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:18:02.278886 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1047/2100] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Aug 24 06:18:02 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:18:02.281447 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4458bece-43ba-4894-b2e5-f7e1734136fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:18:02.282148 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4458bece-43ba-4894-b2e5-f7e1734136fe admin admin] REQUEST_METHOD: `PUT` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:18:02.282457 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4458bece-43ba-4894-b2e5-f7e1734136fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:18:02.282737 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4458bece-43ba-4894-b2e5-f7e1734136fe admin admin] PATH_INFO: `/v3/projects/bdc9217f36d54272b3b5db9a1150eb14/users/30c4582d2fa84c54aa9b42c4e40ba81d/roles/580aa7dbb38d4cff8f9d1bb8983f99ef` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:18:02.283130 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf43944-6d94-49dd-b018-9d8efbf6a216 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:18:02.292675 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf43944-6d94-49dd-b018-9d8efbf6a216 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:18:02.297443 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7843509c-eea5-4c81-9e55-560b07b1736d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:18:02.298077 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7843509c-eea5-4c81-9e55-560b07b1736d admin admin] REQUEST_METHOD: `PUT` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:18:02.298351 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7843509c-eea5-4c81-9e55-560b07b1736d admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:18:02.298620 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7843509c-eea5-4c81-9e55-560b07b1736d admin admin] PATH_INFO: `/v3/projects/cb6af19f57bf4722a87234510e05f70d/users/93d63fb633784adaa2e615b465edc7bd/roles/580aa7dbb38d4cff8f9d1bb8983f99ef` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:18:02.299134 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4458bece-43ba-4894-b2e5-f7e1734136fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=bdc9217f36d54272b3b5db9a1150eb14, user_id=30c4582d2fa84c54aa9b42c4e40ba81d, role_id=580aa7dbb38d4cff8f9d1bb8983f99ef)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:18:02.302113 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4458bece-43ba-4894-b2e5-f7e1734136fe admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:18:02.314540 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7843509c-eea5-4c81-9e55-560b07b1736d admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb6af19f57bf4722a87234510e05f70d, user_id=93d63fb633784adaa2e615b465edc7bd, role_id=580aa7dbb38d4cff8f9d1bb8983f99ef)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:18:02.317239 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7843509c-eea5-4c81-9e55-560b07b1736d admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:18:02.405156 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1054/2101] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Aug 24 06:18:02 2022] PUT /identity/v3/projects/bdc9217f36d54272b3b5db9a1150eb14/users/30c4582d2fa84c54aa9b42c4e40ba81d/roles/580aa7dbb38d4cff8f9d1bb8983f99ef => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 06:18:02.415627 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a38bfc-7910-4423-95e2-a3fe777439fa None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:18:02.415917 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a38bfc-7910-4423-95e2-a3fe777439fa None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:18:02.416196 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a38bfc-7910-4423-95e2-a3fe777439fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:18:02.440278 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1048/2102] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Aug 24 06:18:02 2022] PUT /identity/v3/projects/cb6af19f57bf4722a87234510e05f70d/users/93d63fb633784adaa2e615b465edc7bd/roles/580aa7dbb38d4cff8f9d1bb8983f99ef => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 06:18:02.445002 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-75a38bfc-7910-4423-95e2-a3fe777439fa None None] MFA Rules not processed for user `30c4582d2fa84c54aa9b42c4e40ba81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:18:02.446311 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-75a38bfc-7910-4423-95e2-a3fe777439fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:18:02.451972 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e4bb38-211a-4f7b-86f0-eeba95aee5e2 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:18:02.452260 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e4bb38-211a-4f7b-86f0-eeba95aee5e2 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:18:02.452563 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e4bb38-211a-4f7b-86f0-eeba95aee5e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:18:02.457158 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:18:02.482201 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-99e4bb38-211a-4f7b-86f0-eeba95aee5e2 None None] MFA Rules not processed for user `93d63fb633784adaa2e615b465edc7bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:18:02.483719 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-99e4bb38-211a-4f7b-86f0-eeba95aee5e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:18:02.519574 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1055/2103] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:18:02 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 06:18:02.524444 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:18:02.528639 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a38bfc-7910-4423-95e2-a3fe777439fa None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:18:02.530150 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-75a38bfc-7910-4423-95e2-a3fe777439fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:18:02.549514 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1049/2104] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:18:02 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 06:18:02.554271 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e4bb38-211a-4f7b-86f0-eeba95aee5e2 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:18:02.590369 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a38bfc-7910-4423-95e2-a3fe777439fa None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:18:02.594023 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a04fd84-3b1b-4770-812b-3a82aa538d00 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7d52162cbdc4218b577ecc1e69a88c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:18:02.594655 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a04fd84-3b1b-4770-812b-3a82aa538d00 service glance] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:18:02.594936 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a04fd84-3b1b-4770-812b-3a82aa538d00 service glance] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:18:02.595211 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a04fd84-3b1b-4770-812b-3a82aa538d00 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:18:02.604044 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a04fd84-3b1b-4770-812b-3a82aa538d00 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:18:02.605649 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a04fd84-3b1b-4770-812b-3a82aa538d00 service glance] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:18:02.616064 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1056/2105] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:18:02 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 06:18:02.620479 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e4bb38-211a-4f7b-86f0-eeba95aee5e2 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:18:02.627200 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f14ac8b-109a-4418-a457-811ca45f6b1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:18:02.627810 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f14ac8b-109a-4418-a457-811ca45f6b1d admin admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:18:02.628268 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f14ac8b-109a-4418-a457-811ca45f6b1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:18:02.628636 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f14ac8b-109a-4418-a457-811ca45f6b1d admin admin] PATH_INFO: `/v3/projects` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:18:02.629381 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f14ac8b-109a-4418-a457-811ca45f6b1d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:18:02.631362 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f14ac8b-109a-4418-a457-811ca45f6b1d admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:18:02.730909 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1050/2106] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Aug 24 06:18:02 2022] POST /identity/v3/projects => generated 367 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:18:02.732569 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a04fd84-3b1b-4770-812b-3a82aa538d00 service glance] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:18:02.745372 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a04fd84-3b1b-4770-812b-3a82aa538d00 service glance] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:18:02.820286 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71aa7b36-a9be-441c-9382-59b71cce86fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:18:02.820823 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71aa7b36-a9be-441c-9382-59b71cce86fa admin admin] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:18:02.821108 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71aa7b36-a9be-441c-9382-59b71cce86fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:18:02.821385 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71aa7b36-a9be-441c-9382-59b71cce86fa admin admin] PATH_INFO: `/v3/users` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:18:02.822040 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71aa7b36-a9be-441c-9382-59b71cce86fa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:18:02.824421 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71aa7b36-a9be-441c-9382-59b71cce86fa admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:18:02.919886 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1057/2107] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Aug 24 06:18:02 2022] POST /identity/v3/users => generated 290 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:18:02.924588 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f14ac8b-109a-4418-a457-811ca45f6b1d admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:18:02.935999 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f14ac8b-109a-4418-a457-811ca45f6b1d admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:18:02.940666 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a0039e-a427-42c8-8230-06b52b0bceff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:18:02.941329 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a0039e-a427-42c8-8230-06b52b0bceff admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:18:02.941616 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a0039e-a427-42c8-8230-06b52b0bceff admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:18:02.941896 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a0039e-a427-42c8-8230-06b52b0bceff admin admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:18:02.942541 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a0039e-a427-42c8-8230-06b52b0bceff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:18:02.944152 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a0039e-a427-42c8-8230-06b52b0bceff admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:18:02.958335 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1051/2108] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Aug 24 06:18:02 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:18:02.959697 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71aa7b36-a9be-441c-9382-59b71cce86fa admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:18:02.970134 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71aa7b36-a9be-441c-9382-59b71cce86fa admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:18:02.974766 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1795a4d0-5d10-4bf1-9b37-2e674f9284d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:18:02.975423 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1795a4d0-5d10-4bf1-9b37-2e674f9284d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:18:02.975707 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1795a4d0-5d10-4bf1-9b37-2e674f9284d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:18:02.975981 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1795a4d0-5d10-4bf1-9b37-2e674f9284d3 admin admin] PATH_INFO: `/v3/projects/599efd4a40b24852a3e7eddd015b1f4b/users/172ee764de48413f91b610616226830b/roles/49e47cd157d743f6905f1db07ec878a7` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:18:02.992665 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1795a4d0-5d10-4bf1-9b37-2e674f9284d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=599efd4a40b24852a3e7eddd015b1f4b, user_id=172ee764de48413f91b610616226830b, role_id=49e47cd157d743f6905f1db07ec878a7)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:18:02.995372 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1795a4d0-5d10-4bf1-9b37-2e674f9284d3 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:18:03.080709 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1058/2109] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Aug 24 06:18:02 2022] PUT /identity/v3/projects/599efd4a40b24852a3e7eddd015b1f4b/users/172ee764de48413f91b610616226830b/roles/49e47cd157d743f6905f1db07ec878a7 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 06:18:03.084769 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a0039e-a427-42c8-8230-06b52b0bceff admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:18:03.095383 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a0039e-a427-42c8-8230-06b52b0bceff admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:18:03.160341 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba94f95e-5ce4-45a1-b842-bf8ab9f5782d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:18:03.160987 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba94f95e-5ce4-45a1-b842-bf8ab9f5782d admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:18:03.161393 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba94f95e-5ce4-45a1-b842-bf8ab9f5782d admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:18:03.161675 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba94f95e-5ce4-45a1-b842-bf8ab9f5782d admin admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:18:03.162339 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba94f95e-5ce4-45a1-b842-bf8ab9f5782d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:18:03.163995 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba94f95e-5ce4-45a1-b842-bf8ab9f5782d admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:18:03.175075 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1052/2110] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Aug 24 06:18:03 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:18:03.179459 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1795a4d0-5d10-4bf1-9b37-2e674f9284d3 admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:18:03.191153 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1795a4d0-5d10-4bf1-9b37-2e674f9284d3 admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:18:03.196737 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340e7d02-5c0e-4a13-8568-7806e3d54732 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:18:03.197808 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340e7d02-5c0e-4a13-8568-7806e3d54732 admin admin] REQUEST_METHOD: `PUT` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:18:03.198105 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340e7d02-5c0e-4a13-8568-7806e3d54732 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:18:03.198444 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340e7d02-5c0e-4a13-8568-7806e3d54732 admin admin] PATH_INFO: `/v3/projects/599efd4a40b24852a3e7eddd015b1f4b/users/172ee764de48413f91b610616226830b/roles/49e47cd157d743f6905f1db07ec878a7` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:18:03.201092 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340e7d02-5c0e-4a13-8568-7806e3d54732 admin admin] RBAC: Authorizing `identity:create_grant(project_id=599efd4a40b24852a3e7eddd015b1f4b, user_id=172ee764de48413f91b610616226830b, role_id=49e47cd157d743f6905f1db07ec878a7)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:18:03.204507 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340e7d02-5c0e-4a13-8568-7806e3d54732 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:18:03.218638 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1059/2111] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Aug 24 06:18:03 2022] PUT /identity/v3/projects/599efd4a40b24852a3e7eddd015b1f4b/users/172ee764de48413f91b610616226830b/roles/49e47cd157d743f6905f1db07ec878a7 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:18:03.235303 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e7ed9b-7e9b-411d-83e0-338d0a38a7bb None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:18:03.235787 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e7ed9b-7e9b-411d-83e0-338d0a38a7bb None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:18:03.236138 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e7ed9b-7e9b-411d-83e0-338d0a38a7bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:18:03.267740 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-94e7ed9b-7e9b-411d-83e0-338d0a38a7bb None None] MFA Rules not processed for user `172ee764de48413f91b610616226830b`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:18:03.269254 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-94e7ed9b-7e9b-411d-83e0-338d0a38a7bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:18:03.307763 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:18:03.341287 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1053/2112] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:18:03 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:18:03.352604 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340e7d02-5c0e-4a13-8568-7806e3d54732 admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:18:03.354379 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-340e7d02-5c0e-4a13-8568-7806e3d54732 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:18:03.364665 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:18:03.421758 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340e7d02-5c0e-4a13-8568-7806e3d54732 admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:18:03.426934 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4219e59-fe48-481b-bf82-a5b6116fb2e3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c4def55ac0e48b29836bc5c64bc7402', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:18:03.427585 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4219e59-fe48-481b-bf82-a5b6116fb2e3 service cinder] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:18:03.427861 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4219e59-fe48-481b-bf82-a5b6116fb2e3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:18:03.428131 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4219e59-fe48-481b-bf82-a5b6116fb2e3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:18:03.439444 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4219e59-fe48-481b-bf82-a5b6116fb2e3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:18:03.441067 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4219e59-fe48-481b-bf82-a5b6116fb2e3 service cinder] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:18:03.472662 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1060/2113] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:18:03 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:18:03.702901 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e7ed9b-7e9b-411d-83e0-338d0a38a7bb None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:18:03.718095 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e7ed9b-7e9b-411d-83e0-338d0a38a7bb None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:18:03.722458 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a018239-e680-4715-aa6c-17311a45f335 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c4def55ac0e48b29836bc5c64bc7402', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:18:03.723066 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a018239-e680-4715-aa6c-17311a45f335 service cinder] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:18:03.723346 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a018239-e680-4715-aa6c-17311a45f335 service cinder] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:18:03.723623 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a018239-e680-4715-aa6c-17311a45f335 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:18:03.783378 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a018239-e680-4715-aa6c-17311a45f335 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:18:03.784892 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a018239-e680-4715-aa6c-17311a45f335 service cinder] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:18:03.796989 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1054/2114] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:18:03 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:18:10.694616 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12f54aa-21b0-4e73-8087-e122769b49da None None] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:18:10.695100 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12f54aa-21b0-4e73-8087-e122769b49da None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:18:10.695694 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12f54aa-21b0-4e73-8087-e122769b49da None None] PATH_INFO: `/` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:18:10.697468 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1061/2115] 10.222.0.62 () {54 vars in 921 bytes} [Wed Aug 24 06:18:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:18:10.706265 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233d55a9-a4e6-48d6-ac09-7c7c9fb0ea6a None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:18:10.706802 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233d55a9-a4e6-48d6-ac09-7c7c9fb0ea6a None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:18:10.706802 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233d55a9-a4e6-48d6-ac09-7c7c9fb0ea6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:18:10.733266 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-233d55a9-a4e6-48d6-ac09-7c7c9fb0ea6a None None] MFA Rules not processed for user `abe9d8c9291845dfa4ef55034e5c9e32`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:18:10.749089 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-233d55a9-a4e6-48d6-ac09-7c7c9fb0ea6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:18:10.829891 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1055/2116] 10.222.0.62 () {58 vars in 1018 bytes} [Wed Aug 24 06:18:10 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 06:18:10.840878 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12f54aa-21b0-4e73-8087-e122769b49da None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:18:10.842621 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-c12f54aa-21b0-4e73-8087-e122769b49da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:18:10.860015 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12f54aa-21b0-4e73-8087-e122769b49da None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:18:10.865303 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52190d64-09ec-4d0c-ae36-ff92d7190759 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abe9d8c9291845dfa4ef55034e5c9e32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:18:10.865907 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52190d64-09ec-4d0c-ae36-ff92d7190759 service placement] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:18:10.866186 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52190d64-09ec-4d0c-ae36-ff92d7190759 service placement] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:18:10.866478 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52190d64-09ec-4d0c-ae36-ff92d7190759 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:18:10.875952 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52190d64-09ec-4d0c-ae36-ff92d7190759 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:18:10.877574 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52190d64-09ec-4d0c-ae36-ff92d7190759 service placement] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:18:10.887413 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1062/2117] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:18:10 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:18:10.904202 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233d55a9-a4e6-48d6-ac09-7c7c9fb0ea6a None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:18:10.905837 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-233d55a9-a4e6-48d6-ac09-7c7c9fb0ea6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:18:10.913043 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52190d64-09ec-4d0c-ae36-ff92d7190759 service placement] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:18:11.005319 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52190d64-09ec-4d0c-ae36-ff92d7190759 service placement] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:18:11.007721 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233d55a9-a4e6-48d6-ac09-7c7c9fb0ea6a None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:18:11.012782 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949a339e-87e5-4d1f-a36b-a704ee8241e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:18:11.013632 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949a339e-87e5-4d1f-a36b-a704ee8241e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:18:11.013752 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84a2c6d-2a85-4f02-81c1-e43a17dea93d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:18:11.014082 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949a339e-87e5-4d1f-a36b-a704ee8241e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:18:11.014505 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949a339e-87e5-4d1f-a36b-a704ee8241e8 admin admin] PATH_INFO: `/v3/users/6021a225b5fd4377a9f8120a84232709` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:18:11.015425 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84a2c6d-2a85-4f02-81c1-e43a17dea93d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:18:11.015890 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84a2c6d-2a85-4f02-81c1-e43a17dea93d admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:18:11.016097 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949a339e-87e5-4d1f-a36b-a704ee8241e8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6021a225b5fd4377a9f8120a84232709)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:18:11.016974 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84a2c6d-2a85-4f02-81c1-e43a17dea93d admin admin] PATH_INFO: `/v3/users/93d63fb633784adaa2e615b465edc7bd` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:18:11.018318 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84a2c6d-2a85-4f02-81c1-e43a17dea93d admin admin] RBAC: Authorizing `identity:delete_user(user_id=93d63fb633784adaa2e615b465edc7bd)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:18:11.018552 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949a339e-87e5-4d1f-a36b-a704ee8241e8 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:18:11.020759 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84a2c6d-2a85-4f02-81c1-e43a17dea93d admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:18:11.025688 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:18:11.214398 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-949a339e-87e5-4d1f-a36b-a704ee8241e8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6021a225b5fd4377a9f8120a84232709'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:18:11.222761 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-949a339e-87e5-4d1f-a36b-a704ee8241e8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6021a225b5fd4377a9f8120a84232709'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:18:11.223592 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-949a339e-87e5-4d1f-a36b-a704ee8241e8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6021a225b5fd4377a9f8120a84232709'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:18:11.238798 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-f84a2c6d-2a85-4f02-81c1-e43a17dea93d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '93d63fb633784adaa2e615b465edc7bd'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:18:11.245511 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-f84a2c6d-2a85-4f02-81c1-e43a17dea93d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '93d63fb633784adaa2e615b465edc7bd'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:18:11.246159 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-f84a2c6d-2a85-4f02-81c1-e43a17dea93d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '93d63fb633784adaa2e615b465edc7bd'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:18:11.246761 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-f84a2c6d-2a85-4f02-81c1-e43a17dea93d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '93d63fb633784adaa2e615b465edc7bd'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:18:11.308165 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-949a339e-87e5-4d1f-a36b-a704ee8241e8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6021a225b5fd4377a9f8120a84232709'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:18:11.324171 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-949a339e-87e5-4d1f-a36b-a704ee8241e8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6021a225b5fd4377a9f8120a84232709'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:18:11.330915 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1056/2118] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Aug 24 06:18:10 2022] DELETE /identity/v3/users/6021a225b5fd4377a9f8120a84232709 => generated 0 bytes in 427 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:18:11.353608 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-f84a2c6d-2a85-4f02-81c1-e43a17dea93d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '93d63fb633784adaa2e615b465edc7bd'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:18:11.405353 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1063/2119] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Aug 24 06:18:10 2022] DELETE /identity/v3/users/93d63fb633784adaa2e615b465edc7bd => generated 0 bytes in 489 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 06:18:11.407490 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17a488a-4d21-4122-afa8-0e01f7171318 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:18:11.407774 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17a488a-4d21-4122-afa8-0e01f7171318 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:18:11.408052 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17a488a-4d21-4122-afa8-0e01f7171318 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:18:11.428850 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-f17a488a-4d21-4122-afa8-0e01f7171318 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:18:11.436300 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-f17a488a-4d21-4122-afa8-0e01f7171318 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:18:11.520268 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1057/2120] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:18:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:18:11.531555 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84a2c6d-2a85-4f02-81c1-e43a17dea93d admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:18:11.533198 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-f84a2c6d-2a85-4f02-81c1-e43a17dea93d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:18:11.556297 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:18:11.605929 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84a2c6d-2a85-4f02-81c1-e43a17dea93d admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:18:11.610688 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f35288-31c3-43dd-8a4b-ec4cf4e1c846 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bb6c79db75e4d169331c02133d1ec32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:18:11.611294 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f35288-31c3-43dd-8a4b-ec4cf4e1c846 service neutron] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:18:11.611567 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f35288-31c3-43dd-8a4b-ec4cf4e1c846 service neutron] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:18:11.611836 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f35288-31c3-43dd-8a4b-ec4cf4e1c846 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:18:11.620850 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f35288-31c3-43dd-8a4b-ec4cf4e1c846 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:18:11.622272 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f35288-31c3-43dd-8a4b-ec4cf4e1c846 service neutron] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:18:11.632226 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1064/2121] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:18:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:18:11.709623 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17a488a-4d21-4122-afa8-0e01f7171318 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:18:11.711227 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-f17a488a-4d21-4122-afa8-0e01f7171318 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:18:11.724669 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17a488a-4d21-4122-afa8-0e01f7171318 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:18:11.730480 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e26ec5-fbd5-4137-8090-dec93f3831c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:18:11.731249 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e26ec5-fbd5-4137-8090-dec93f3831c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:18:11.731595 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e26ec5-fbd5-4137-8090-dec93f3831c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:18:11.731931 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e26ec5-fbd5-4137-8090-dec93f3831c8 admin admin] PATH_INFO: `/v3/projects/ed8aa953e25943a197bec8ac5148a39c` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:18:11.733339 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e26ec5-fbd5-4137-8090-dec93f3831c8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ed8aa953e25943a197bec8ac5148a39c)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:18:11.735805 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e26ec5-fbd5-4137-8090-dec93f3831c8 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:18:11.853750 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-f1e26ec5-fbd5-4137-8090-dec93f3831c8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ed8aa953e25943a197bec8ac5148a39c', 'request_id': 'req-f1e26ec5-fbd5-4137-8090-dec93f3831c8'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:18:11.861335 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-f1e26ec5-fbd5-4137-8090-dec93f3831c8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ed8aa953e25943a197bec8ac5148a39c', 'request_id': 'req-f1e26ec5-fbd5-4137-8090-dec93f3831c8'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:18:11.867661 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-f1e26ec5-fbd5-4137-8090-dec93f3831c8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ed8aa953e25943a197bec8ac5148a39c', 'request_id': 'req-f1e26ec5-fbd5-4137-8090-dec93f3831c8'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:18:11.926056 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-f1e26ec5-fbd5-4137-8090-dec93f3831c8 admin admin] The token cache is being invalidate because project ed8aa953e25943a197bec8ac5148a39c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81852) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:18:11.926463 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-f1e26ec5-fbd5-4137-8090-dec93f3831c8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:18:11.927194 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-f1e26ec5-fbd5-4137-8090-dec93f3831c8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:18:11.928852 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1058/2122] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Aug 24 06:18:11 2022] DELETE /identity/v3/projects/ed8aa953e25943a197bec8ac5148a39c => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 06:18:11.933861 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb74a31-b4f2-41dd-8de7-5045963a2a88 None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:18:11.934447 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb74a31-b4f2-41dd-8de7-5045963a2a88 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:18:11.934447 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb74a31-b4f2-41dd-8de7-5045963a2a88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:18:11.956237 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-2fb74a31-b4f2-41dd-8de7-5045963a2a88 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:18:11.960604 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-2fb74a31-b4f2-41dd-8de7-5045963a2a88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:18:12.022818 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1065/2123] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:18:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:18:12.031322 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e4df47-767b-4b07-a790-16f35340b9a5 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:18:12.032616 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e4df47-767b-4b07-a790-16f35340b9a5 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:18:12.032616 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e4df47-767b-4b07-a790-16f35340b9a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:18:12.039377 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb74a31-b4f2-41dd-8de7-5045963a2a88 None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:18:12.045439 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-2fb74a31-b4f2-41dd-8de7-5045963a2a88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:18:12.060735 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-e2e4df47-767b-4b07-a790-16f35340b9a5 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:18:12.061953 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-e2e4df47-767b-4b07-a790-16f35340b9a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:18:12.067919 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb74a31-b4f2-41dd-8de7-5045963a2a88 None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:18:12.076371 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e59dbfe-db3b-42c8-97fd-e75fcbc4b126 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:18:12.077687 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1059/2124] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:18:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:18:12.081065 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e4df47-767b-4b07-a790-16f35340b9a5 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:18:12.081259 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e59dbfe-db3b-42c8-97fd-e75fcbc4b126 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:18:12.081616 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e59dbfe-db3b-42c8-97fd-e75fcbc4b126 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:18:12.081959 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e59dbfe-db3b-42c8-97fd-e75fcbc4b126 admin admin] PATH_INFO: `/v3/projects/cb6af19f57bf4722a87234510e05f70d` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:18:12.083586 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e59dbfe-db3b-42c8-97fd-e75fcbc4b126 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cb6af19f57bf4722a87234510e05f70d)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:18:12.085259 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e59dbfe-db3b-42c8-97fd-e75fcbc4b126 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:18:12.086165 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:18:12.097335 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e4df47-767b-4b07-a790-16f35340b9a5 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:18:12.103626 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138ca657-de2e-4375-b6d9-aa1a698ee394 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:18:12.104348 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138ca657-de2e-4375-b6d9-aa1a698ee394 admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:18:12.104721 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138ca657-de2e-4375-b6d9-aa1a698ee394 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:18:12.105169 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138ca657-de2e-4375-b6d9-aa1a698ee394 admin admin] PATH_INFO: `/v3/domains` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:18:12.106651 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138ca657-de2e-4375-b6d9-aa1a698ee394 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:18:12.106651 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138ca657-de2e-4375-b6d9-aa1a698ee394 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:18:12.107658 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138ca657-de2e-4375-b6d9-aa1a698ee394 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:18:12.119789 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1060/2125] 10.222.0.62 () {58 vars in 1153 bytes} [Wed Aug 24 06:18:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 06:18:12.124201 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138ca657-de2e-4375-b6d9-aa1a698ee394 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:18:12.134758 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138ca657-de2e-4375-b6d9-aa1a698ee394 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:18:12.139349 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e57eab7-e207-4acc-8d61-b52fdeb68f34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:18:12.139919 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e57eab7-e207-4acc-8d61-b52fdeb68f34 admin admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:18:12.140197 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e57eab7-e207-4acc-8d61-b52fdeb68f34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:18:12.140468 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e57eab7-e207-4acc-8d61-b52fdeb68f34 admin admin] PATH_INFO: `/v3/projects` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:18:12.141707 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e57eab7-e207-4acc-8d61-b52fdeb68f34 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:18:12.143598 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e57eab7-e207-4acc-8d61-b52fdeb68f34 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:18:12.212051 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-2e59dbfe-db3b-42c8-97fd-e75fcbc4b126 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cb6af19f57bf4722a87234510e05f70d', 'request_id': 'req-2e59dbfe-db3b-42c8-97fd-e75fcbc4b126'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:18:12.218558 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-2e59dbfe-db3b-42c8-97fd-e75fcbc4b126 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cb6af19f57bf4722a87234510e05f70d', 'request_id': 'req-2e59dbfe-db3b-42c8-97fd-e75fcbc4b126'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:18:12.226463 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-2e59dbfe-db3b-42c8-97fd-e75fcbc4b126 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cb6af19f57bf4722a87234510e05f70d', 'request_id': 'req-2e59dbfe-db3b-42c8-97fd-e75fcbc4b126'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:18:12.284056 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1061/2126] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Aug 24 06:18:12 2022] POST /identity/v3/projects => generated 391 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 06:18:12.285150 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e57eab7-e207-4acc-8d61-b52fdeb68f34 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:18:12.320954 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-2e59dbfe-db3b-42c8-97fd-e75fcbc4b126 admin admin] The token cache is being invalidate because project cb6af19f57bf4722a87234510e05f70d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81851) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:18:12.321389 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-2e59dbfe-db3b-42c8-97fd-e75fcbc4b126 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:18:12.322184 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-2e59dbfe-db3b-42c8-97fd-e75fcbc4b126 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:18:12.331404 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1066/2127] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Aug 24 06:18:12 2022] DELETE /identity/v3/projects/cb6af19f57bf4722a87234510e05f70d => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:18:12.332547 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e59dbfe-db3b-42c8-97fd-e75fcbc4b126 admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:18:12.334438 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-2e59dbfe-db3b-42c8-97fd-e75fcbc4b126 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:18:12.362497 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e57eab7-e207-4acc-8d61-b52fdeb68f34 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:18:12.367700 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25fd2ea-d93c-4dbd-b7ed-4dee2bc6b6f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:18:12.368343 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25fd2ea-d93c-4dbd-b7ed-4dee2bc6b6f7 admin admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:18:12.368626 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25fd2ea-d93c-4dbd-b7ed-4dee2bc6b6f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:18:12.368947 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25fd2ea-d93c-4dbd-b7ed-4dee2bc6b6f7 admin admin] PATH_INFO: `/v3/users` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:18:12.369666 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25fd2ea-d93c-4dbd-b7ed-4dee2bc6b6f7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:18:12.371992 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25fd2ea-d93c-4dbd-b7ed-4dee2bc6b6f7 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:18:12.394070 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:18:12.413915 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e59dbfe-db3b-42c8-97fd-e75fcbc4b126 admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:18:12.418662 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926cde64-941c-43c9-bad2-fc397f7d1613 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:18:12.419331 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926cde64-941c-43c9-bad2-fc397f7d1613 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:18:12.419610 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926cde64-941c-43c9-bad2-fc397f7d1613 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:18:12.419931 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926cde64-941c-43c9-bad2-fc397f7d1613 admin admin] PATH_INFO: `/v3/users/172ee764de48413f91b610616226830b` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:18:12.421177 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926cde64-941c-43c9-bad2-fc397f7d1613 admin admin] RBAC: Authorizing `identity:delete_user(user_id=172ee764de48413f91b610616226830b)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:18:12.423619 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926cde64-941c-43c9-bad2-fc397f7d1613 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:18:12.485938 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1062/2128] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Aug 24 06:18:12 2022] POST /identity/v3/users => generated 296 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 06:18:12.490032 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25fd2ea-d93c-4dbd-b7ed-4dee2bc6b6f7 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:18:12.491624 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-c25fd2ea-d93c-4dbd-b7ed-4dee2bc6b6f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:18:12.504158 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25fd2ea-d93c-4dbd-b7ed-4dee2bc6b6f7 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:18:12.509607 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebc67c4-a862-4667-a61c-6102e3dc3bda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:18:12.509777 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebc67c4-a862-4667-a61c-6102e3dc3bda admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:18:12.510058 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebc67c4-a862-4667-a61c-6102e3dc3bda admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:18:12.510330 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebc67c4-a862-4667-a61c-6102e3dc3bda admin admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:18:12.511015 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebc67c4-a862-4667-a61c-6102e3dc3bda admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:18:12.513377 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebc67c4-a862-4667-a61c-6102e3dc3bda admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:18:12.524614 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1063/2129] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Aug 24 06:18:12 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 06:18:12.528736 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebc67c4-a862-4667-a61c-6102e3dc3bda admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:18:12.539965 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebc67c4-a862-4667-a61c-6102e3dc3bda admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:18:12.544777 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1554a036-7e1e-43f6-ac73-24db3780b345 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:18:12.545379 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1554a036-7e1e-43f6-ac73-24db3780b345 admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:18:12.545716 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1554a036-7e1e-43f6-ac73-24db3780b345 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:18:12.546115 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1554a036-7e1e-43f6-ac73-24db3780b345 admin admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:18:12.546705 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1554a036-7e1e-43f6-ac73-24db3780b345 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:18:12.548834 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1554a036-7e1e-43f6-ac73-24db3780b345 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:18:12.560042 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1064/2130] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Aug 24 06:18:12 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 06:18:12.564704 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1554a036-7e1e-43f6-ac73-24db3780b345 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:18:12.576527 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1554a036-7e1e-43f6-ac73-24db3780b345 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:18:12.583304 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4c2c8f-26b6-45de-9113-9a28c6e9f26f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:18:12.583974 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4c2c8f-26b6-45de-9113-9a28c6e9f26f admin admin] REQUEST_METHOD: `PUT` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:18:12.584251 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4c2c8f-26b6-45de-9113-9a28c6e9f26f admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:18:12.584570 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4c2c8f-26b6-45de-9113-9a28c6e9f26f admin admin] PATH_INFO: `/v3/projects/c467d980c962456c9fa0e28364efd2a3/users/069dc6f771a74dd4ab22d9d5e68e3cc9/roles/580aa7dbb38d4cff8f9d1bb8983f99ef` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:18:12.609978 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4c2c8f-26b6-45de-9113-9a28c6e9f26f admin admin] RBAC: Authorizing `identity:create_grant(project_id=c467d980c962456c9fa0e28364efd2a3, user_id=069dc6f771a74dd4ab22d9d5e68e3cc9, role_id=580aa7dbb38d4cff8f9d1bb8983f99ef)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:18:12.616134 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4c2c8f-26b6-45de-9113-9a28c6e9f26f admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:18:12.691450 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-926cde64-941c-43c9-bad2-fc397f7d1613 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '172ee764de48413f91b610616226830b'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:18:12.699726 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-926cde64-941c-43c9-bad2-fc397f7d1613 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '172ee764de48413f91b610616226830b'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:18:12.700619 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-926cde64-941c-43c9-bad2-fc397f7d1613 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '172ee764de48413f91b610616226830b'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:18:12.705616 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-926cde64-941c-43c9-bad2-fc397f7d1613 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '172ee764de48413f91b610616226830b'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:18:12.707563 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1065/2131] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Aug 24 06:18:12 2022] PUT /identity/v3/projects/c467d980c962456c9fa0e28364efd2a3/users/069dc6f771a74dd4ab22d9d5e68e3cc9/roles/580aa7dbb38d4cff8f9d1bb8983f99ef => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 06:18:12.712760 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf236d5b-a488-47dd-a526-07cbd6361449 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:18:12.713376 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf236d5b-a488-47dd-a526-07cbd6361449 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:18:12.713376 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf236d5b-a488-47dd-a526-07cbd6361449 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:18:12.736724 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-bf236d5b-a488-47dd-a526-07cbd6361449 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:18:12.738391 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-bf236d5b-a488-47dd-a526-07cbd6361449 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:18:12.791659 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-926cde64-941c-43c9-bad2-fc397f7d1613 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '172ee764de48413f91b610616226830b'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:18:12.815085 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1066/2132] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:18:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:18:12.816726 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1067/2133] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Aug 24 06:18:12 2022] DELETE /identity/v3/users/172ee764de48413f91b610616226830b => generated 0 bytes in 485 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:18:12.824974 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf236d5b-a488-47dd-a526-07cbd6361449 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:18:12.826592 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-bf236d5b-a488-47dd-a526-07cbd6361449 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:18:12.845893 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:18:12.918443 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf236d5b-a488-47dd-a526-07cbd6361449 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:18:12.979274 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed361950-7335-4296-825b-903e660a762e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bb6c79db75e4d169331c02133d1ec32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:18:12.980608 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed361950-7335-4296-825b-903e660a762e service neutron] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:18:12.981144 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed361950-7335-4296-825b-903e660a762e service neutron] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:18:12.981276 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed361950-7335-4296-825b-903e660a762e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:18:13.076042 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed361950-7335-4296-825b-903e660a762e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:18:13.077830 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed361950-7335-4296-825b-903e660a762e service neutron] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:18:13.088863 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1067/2134] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:18:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 264 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 06:18:13.282030 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926cde64-941c-43c9-bad2-fc397f7d1613 admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:18:13.283558 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-926cde64-941c-43c9-bad2-fc397f7d1613 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:18:13.298062 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926cde64-941c-43c9-bad2-fc397f7d1613 admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:18:13.304363 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c372ee-1969-41b1-a8be-85343cf152f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:18:13.306746 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c372ee-1969-41b1-a8be-85343cf152f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:18:13.306746 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c372ee-1969-41b1-a8be-85343cf152f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:18:13.306746 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c372ee-1969-41b1-a8be-85343cf152f7 admin admin] PATH_INFO: `/v3/projects/599efd4a40b24852a3e7eddd015b1f4b` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:18:13.306746 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c372ee-1969-41b1-a8be-85343cf152f7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=599efd4a40b24852a3e7eddd015b1f4b)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:18:13.308466 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c372ee-1969-41b1-a8be-85343cf152f7 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:18:13.411668 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-71c372ee-1969-41b1-a8be-85343cf152f7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '599efd4a40b24852a3e7eddd015b1f4b', 'request_id': 'req-71c372ee-1969-41b1-a8be-85343cf152f7'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:18:13.417181 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-71c372ee-1969-41b1-a8be-85343cf152f7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '599efd4a40b24852a3e7eddd015b1f4b', 'request_id': 'req-71c372ee-1969-41b1-a8be-85343cf152f7'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:18:13.423452 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-71c372ee-1969-41b1-a8be-85343cf152f7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '599efd4a40b24852a3e7eddd015b1f4b', 'request_id': 'req-71c372ee-1969-41b1-a8be-85343cf152f7'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:18:13.484812 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-71c372ee-1969-41b1-a8be-85343cf152f7 admin admin] The token cache is being invalidate because project 599efd4a40b24852a3e7eddd015b1f4b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81851) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:18:13.485189 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-71c372ee-1969-41b1-a8be-85343cf152f7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:18:13.485962 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-71c372ee-1969-41b1-a8be-85343cf152f7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:18:13.487661 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1068/2135] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Aug 24 06:18:13 2022] DELETE /identity/v3/projects/599efd4a40b24852a3e7eddd015b1f4b => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:18:13.492721 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bfe930-a2eb-48db-a17a-5c38cbf05f47 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:18:13.493007 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bfe930-a2eb-48db-a17a-5c38cbf05f47 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:18:13.493289 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bfe930-a2eb-48db-a17a-5c38cbf05f47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:18:13.514573 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-d5bfe930-a2eb-48db-a17a-5c38cbf05f47 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:18:13.516532 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-d5bfe930-a2eb-48db-a17a-5c38cbf05f47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:18:13.579005 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1068/2136] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:18:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:18:13.587115 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c949c0-5fe8-49ee-80d4-0914aea13640 None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:18:13.587292 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c949c0-5fe8-49ee-80d4-0914aea13640 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:18:13.587585 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c949c0-5fe8-49ee-80d4-0914aea13640 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:18:13.608375 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-44c949c0-5fe8-49ee-80d4-0914aea13640 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:18:13.609697 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-44c949c0-5fe8-49ee-80d4-0914aea13640 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:18:13.618713 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1069/2137] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:18:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:18:13.623443 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bfe930-a2eb-48db-a17a-5c38cbf05f47 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:18:13.634169 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bfe930-a2eb-48db-a17a-5c38cbf05f47 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:18:13.638859 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b74c20a-489f-451f-b311-ddc78c802b99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:18:13.639473 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b74c20a-489f-451f-b311-ddc78c802b99 admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:18:13.639758 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b74c20a-489f-451f-b311-ddc78c802b99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:18:13.640033 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b74c20a-489f-451f-b311-ddc78c802b99 admin admin] PATH_INFO: `/v3/domains` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:18:13.641286 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b74c20a-489f-451f-b311-ddc78c802b99 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:18:13.641586 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b74c20a-489f-451f-b311-ddc78c802b99 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:18:13.643240 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b74c20a-489f-451f-b311-ddc78c802b99 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:18:13.655512 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1069/2138] 10.222.0.62 () {58 vars in 1153 bytes} [Wed Aug 24 06:18:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:18:13.660049 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c949c0-5fe8-49ee-80d4-0914aea13640 None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:18:13.669792 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c949c0-5fe8-49ee-80d4-0914aea13640 None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:18:13.676019 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816c8eb2-052b-4282-bece-f5eb1b5411cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:18:13.676780 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816c8eb2-052b-4282-bece-f5eb1b5411cc admin admin] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:18:13.677064 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816c8eb2-052b-4282-bece-f5eb1b5411cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:18:13.677340 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816c8eb2-052b-4282-bece-f5eb1b5411cc admin admin] PATH_INFO: `/v3/projects` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:18:13.678041 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816c8eb2-052b-4282-bece-f5eb1b5411cc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:18:13.682291 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816c8eb2-052b-4282-bece-f5eb1b5411cc admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:18:13.806317 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1070/2139] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Aug 24 06:18:13 2022] POST /identity/v3/projects => generated 377 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:18:13.806420 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b74c20a-489f-451f-b311-ddc78c802b99 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:18:13.813055 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:18:13.817735 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b74c20a-489f-451f-b311-ddc78c802b99 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:18:13.889101 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d9751e-5302-4364-ad8b-e6253d540e80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:18:13.889713 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d9751e-5302-4364-ad8b-e6253d540e80 admin admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:18:13.891930 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d9751e-5302-4364-ad8b-e6253d540e80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:18:13.891930 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d9751e-5302-4364-ad8b-e6253d540e80 admin admin] PATH_INFO: `/v3/users` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:18:13.891930 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d9751e-5302-4364-ad8b-e6253d540e80 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:18:13.893505 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d9751e-5302-4364-ad8b-e6253d540e80 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:18:13.994748 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1070/2140] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Aug 24 06:18:13 2022] POST /identity/v3/users => generated 289 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 06:18:13.999010 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816c8eb2-052b-4282-bece-f5eb1b5411cc admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:18:14.013157 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816c8eb2-052b-4282-bece-f5eb1b5411cc admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:18:14.032139 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520d6fae-68f1-4070-bc36-49a849d53c2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:18:14.032801 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520d6fae-68f1-4070-bc36-49a849d53c2d admin admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:18:14.033205 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520d6fae-68f1-4070-bc36-49a849d53c2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:18:14.033360 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520d6fae-68f1-4070-bc36-49a849d53c2d admin admin] PATH_INFO: `/v3/roles` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:18:14.034073 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520d6fae-68f1-4070-bc36-49a849d53c2d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:18:14.035693 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520d6fae-68f1-4070-bc36-49a849d53c2d admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:18:14.046430 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1071/2141] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Aug 24 06:18:13 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:18:14.051415 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d9751e-5302-4364-ad8b-e6253d540e80 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:18:14.064260 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d9751e-5302-4364-ad8b-e6253d540e80 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:18:14.069840 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17239234-9f8c-4872-b06e-34b30dcf80f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:18:14.070905 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17239234-9f8c-4872-b06e-34b30dcf80f5 admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:18:14.070905 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17239234-9f8c-4872-b06e-34b30dcf80f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:18:14.071196 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17239234-9f8c-4872-b06e-34b30dcf80f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:18:14.071659 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17239234-9f8c-4872-b06e-34b30dcf80f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:18:14.073362 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17239234-9f8c-4872-b06e-34b30dcf80f5 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:18:14.086432 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1071/2142] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Aug 24 06:18:14 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:18:14.090648 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520d6fae-68f1-4070-bc36-49a849d53c2d admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:18:14.101547 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=80,keystone:DELETE=17,keystone:INSERT=3 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:18:14.106573 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520d6fae-68f1-4070-bc36-49a849d53c2d admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:18:14.112215 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f412a587-22f3-4938-aed9-d0cfa35cc9bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:18:14.113021 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f412a587-22f3-4938-aed9-d0cfa35cc9bc admin admin] REQUEST_METHOD: `PUT` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:18:14.113302 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f412a587-22f3-4938-aed9-d0cfa35cc9bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:18:14.120100 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f412a587-22f3-4938-aed9-d0cfa35cc9bc admin admin] PATH_INFO: `/v3/projects/ded654ad081f4766b5a287ad5c4a5758/users/e81a5462bd354f25bc4d237057264729/roles/580aa7dbb38d4cff8f9d1bb8983f99ef` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:18:14.130499 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f412a587-22f3-4938-aed9-d0cfa35cc9bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=ded654ad081f4766b5a287ad5c4a5758, user_id=e81a5462bd354f25bc4d237057264729, role_id=580aa7dbb38d4cff8f9d1bb8983f99ef)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:18:14.133993 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f412a587-22f3-4938-aed9-d0cfa35cc9bc admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:18:14.194607 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1072/2143] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Aug 24 06:18:14 2022] PUT /identity/v3/projects/ded654ad081f4766b5a287ad5c4a5758/users/e81a5462bd354f25bc4d237057264729/roles/580aa7dbb38d4cff8f9d1bb8983f99ef => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 06:18:14.211258 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6a9c8b-3c14-4ebf-8393-46fdb4e96a0e None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:18:14.211816 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6a9c8b-3c14-4ebf-8393-46fdb4e96a0e None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:18:14.212159 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6a9c8b-3c14-4ebf-8393-46fdb4e96a0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:18:14.241657 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-4d6a9c8b-3c14-4ebf-8393-46fdb4e96a0e None None] MFA Rules not processed for user `e81a5462bd354f25bc4d237057264729`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:18:14.242997 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-4d6a9c8b-3c14-4ebf-8393-46fdb4e96a0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:18:14.339979 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1072/2144] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:18:14 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 06:18:14.347250 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f412a587-22f3-4938-aed9-d0cfa35cc9bc admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:18:14.424037 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f412a587-22f3-4938-aed9-d0cfa35cc9bc admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:18:14.429373 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac6a63f-5e42-4bc5-935f-a6e719407bb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:18:14.430212 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac6a63f-5e42-4bc5-935f-a6e719407bb2 admin admin] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:18:14.430627 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac6a63f-5e42-4bc5-935f-a6e719407bb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:18:14.431223 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac6a63f-5e42-4bc5-935f-a6e719407bb2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:18:14.432228 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac6a63f-5e42-4bc5-935f-a6e719407bb2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:18:14.434703 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac6a63f-5e42-4bc5-935f-a6e719407bb2 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:18:14.533919 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1073/2145] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Aug 24 06:18:14 2022] POST /identity/v3/projects => generated 377 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 06:18:14.538031 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6a9c8b-3c14-4ebf-8393-46fdb4e96a0e None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:18:14.551873 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6a9c8b-3c14-4ebf-8393-46fdb4e96a0e None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:18:14.629512 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee6aa38-a122-4ecc-92b5-dea09a93e314 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:18:14.630801 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee6aa38-a122-4ecc-92b5-dea09a93e314 admin admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:18:14.635754 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee6aa38-a122-4ecc-92b5-dea09a93e314 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:18:14.636190 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee6aa38-a122-4ecc-92b5-dea09a93e314 admin admin] PATH_INFO: `/v3/users` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:18:14.637046 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee6aa38-a122-4ecc-92b5-dea09a93e314 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:18:14.639672 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee6aa38-a122-4ecc-92b5-dea09a93e314 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:18:14.776617 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1073/2146] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Aug 24 06:18:14 2022] POST /identity/v3/users => generated 289 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 06:18:14.779574 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac6a63f-5e42-4bc5-935f-a6e719407bb2 admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:18:14.795871 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac6a63f-5e42-4bc5-935f-a6e719407bb2 admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:18:14.807084 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1de111-334a-4c62-9bd4-f5abb732701a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:18:14.807800 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1de111-334a-4c62-9bd4-f5abb732701a admin admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:18:14.808091 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1de111-334a-4c62-9bd4-f5abb732701a admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:18:14.808366 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1de111-334a-4c62-9bd4-f5abb732701a admin admin] PATH_INFO: `/v3/roles` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:18:14.809071 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1de111-334a-4c62-9bd4-f5abb732701a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:18:14.810708 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1de111-334a-4c62-9bd4-f5abb732701a admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:18:14.832437 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1074/2147] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Aug 24 06:18:14 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:18:14.836327 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee6aa38-a122-4ecc-92b5-dea09a93e314 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:18:14.842035 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:18:14.849456 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee6aa38-a122-4ecc-92b5-dea09a93e314 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:18:14.854432 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaeb73d5-5a29-4752-b890-ae83177fc128 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:18:14.855236 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaeb73d5-5a29-4752-b890-ae83177fc128 admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:18:14.855484 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaeb73d5-5a29-4752-b890-ae83177fc128 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:18:14.860112 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaeb73d5-5a29-4752-b890-ae83177fc128 admin admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:18:14.860112 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaeb73d5-5a29-4752-b890-ae83177fc128 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:18:14.860112 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaeb73d5-5a29-4752-b890-ae83177fc128 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:18:14.870921 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1074/2148] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Aug 24 06:18:14 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:18:14.873497 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1de111-334a-4c62-9bd4-f5abb732701a admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:18:14.892390 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1de111-334a-4c62-9bd4-f5abb732701a admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:18:14.897421 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0daba1-c4d2-4d0e-b539-cd183f51549f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:18:14.898105 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0daba1-c4d2-4d0e-b539-cd183f51549f admin admin] REQUEST_METHOD: `PUT` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:18:14.898376 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0daba1-c4d2-4d0e-b539-cd183f51549f admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:18:14.898645 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0daba1-c4d2-4d0e-b539-cd183f51549f admin admin] PATH_INFO: `/v3/projects/6a4243fe591b4947933d358531a20c55/users/a0443372b1c14d8085c531484e4c516c/roles/580aa7dbb38d4cff8f9d1bb8983f99ef` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:18:14.940746 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0daba1-c4d2-4d0e-b539-cd183f51549f admin admin] RBAC: Authorizing `identity:create_grant(project_id=6a4243fe591b4947933d358531a20c55, user_id=a0443372b1c14d8085c531484e4c516c, role_id=580aa7dbb38d4cff8f9d1bb8983f99ef)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:18:14.944057 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0daba1-c4d2-4d0e-b539-cd183f51549f admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:18:15.040378 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1075/2149] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Aug 24 06:18:14 2022] PUT /identity/v3/projects/6a4243fe591b4947933d358531a20c55/users/a0443372b1c14d8085c531484e4c516c/roles/580aa7dbb38d4cff8f9d1bb8983f99ef => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:18:15.058629 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4a2c62-7c1f-46dc-8912-ac841d2a7f76 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:18:15.058991 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4a2c62-7c1f-46dc-8912-ac841d2a7f76 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:18:15.059338 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4a2c62-7c1f-46dc-8912-ac841d2a7f76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:18:15.096243 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-9e4a2c62-7c1f-46dc-8912-ac841d2a7f76 None None] MFA Rules not processed for user `a0443372b1c14d8085c531484e4c516c`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:18:15.097717 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-9e4a2c62-7c1f-46dc-8912-ac841d2a7f76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:18:15.208193 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1075/2150] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:18:15 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:18:15.212903 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0daba1-c4d2-4d0e-b539-cd183f51549f admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:18:15.298005 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0daba1-c4d2-4d0e-b539-cd183f51549f admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:18:15.302217 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8218b9bf-a9c4-4804-8d99-ca319a9f427e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:18:15.302842 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8218b9bf-a9c4-4804-8d99-ca319a9f427e admin admin] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:18:15.303124 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8218b9bf-a9c4-4804-8d99-ca319a9f427e admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:18:15.303399 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8218b9bf-a9c4-4804-8d99-ca319a9f427e admin admin] PATH_INFO: `/v3/projects` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:18:15.304050 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8218b9bf-a9c4-4804-8d99-ca319a9f427e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:18:15.306117 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8218b9bf-a9c4-4804-8d99-ca319a9f427e admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:18:15.413298 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1076/2151] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Aug 24 06:18:15 2022] POST /identity/v3/projects => generated 375 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:18:15.415096 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4a2c62-7c1f-46dc-8912-ac841d2a7f76 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:18:15.435415 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4a2c62-7c1f-46dc-8912-ac841d2a7f76 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:18:15.522374 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16edc14b-f4fc-4127-885b-b0cd13c7261e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:18:15.523050 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16edc14b-f4fc-4127-885b-b0cd13c7261e admin admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:18:15.523399 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16edc14b-f4fc-4127-885b-b0cd13c7261e admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:18:15.523727 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16edc14b-f4fc-4127-885b-b0cd13c7261e admin admin] PATH_INFO: `/v3/users` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:18:15.524364 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16edc14b-f4fc-4127-885b-b0cd13c7261e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:18:15.526826 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16edc14b-f4fc-4127-885b-b0cd13c7261e admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:18:15.668726 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1076/2152] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Aug 24 06:18:15 2022] POST /identity/v3/users => generated 294 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 06:18:15.672736 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8218b9bf-a9c4-4804-8d99-ca319a9f427e admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:18:15.696863 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8218b9bf-a9c4-4804-8d99-ca319a9f427e admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:18:15.701755 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc00e7a-e849-4d3e-b406-f26cefcb4c8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:18:15.708367 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc00e7a-e849-4d3e-b406-f26cefcb4c8b admin admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:18:15.708367 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc00e7a-e849-4d3e-b406-f26cefcb4c8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:18:15.708367 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc00e7a-e849-4d3e-b406-f26cefcb4c8b admin admin] PATH_INFO: `/v3/roles` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:18:15.708367 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc00e7a-e849-4d3e-b406-f26cefcb4c8b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:18:15.708367 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc00e7a-e849-4d3e-b406-f26cefcb4c8b admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:18:15.723217 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1077/2153] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Aug 24 06:18:15 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:18:15.726714 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16edc14b-f4fc-4127-885b-b0cd13c7261e admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:18:15.738960 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16edc14b-f4fc-4127-885b-b0cd13c7261e admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:18:15.745268 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc6e96b-02a1-40f8-8782-dc5cf0b4bc8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:18:15.746097 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc6e96b-02a1-40f8-8782-dc5cf0b4bc8f admin admin] REQUEST_METHOD: `PUT` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:18:15.746787 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc6e96b-02a1-40f8-8782-dc5cf0b4bc8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:18:15.755167 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc6e96b-02a1-40f8-8782-dc5cf0b4bc8f admin admin] PATH_INFO: `/v3/projects/f1a762f10d5342c4a34affa811d060ee/users/995c6173024d4a02ab322e1cd1e4a742/roles/49e47cd157d743f6905f1db07ec878a7` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:18:15.774222 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc6e96b-02a1-40f8-8782-dc5cf0b4bc8f admin admin] RBAC: Authorizing `identity:create_grant(project_id=f1a762f10d5342c4a34affa811d060ee, user_id=995c6173024d4a02ab322e1cd1e4a742, role_id=49e47cd157d743f6905f1db07ec878a7)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:18:15.777165 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc6e96b-02a1-40f8-8782-dc5cf0b4bc8f admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:18:15.894448 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1077/2154] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Aug 24 06:18:15 2022] PUT /identity/v3/projects/f1a762f10d5342c4a34affa811d060ee/users/995c6173024d4a02ab322e1cd1e4a742/roles/49e47cd157d743f6905f1db07ec878a7 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:18:15.895496 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc00e7a-e849-4d3e-b406-f26cefcb4c8b admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:18:15.901917 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:18:15.909387 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc00e7a-e849-4d3e-b406-f26cefcb4c8b admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:18:15.994975 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f01c5a-7182-4af0-b363-d88c2dbb421d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:18:15.995594 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f01c5a-7182-4af0-b363-d88c2dbb421d admin admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:18:15.995875 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f01c5a-7182-4af0-b363-d88c2dbb421d admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:18:15.996150 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f01c5a-7182-4af0-b363-d88c2dbb421d admin admin] PATH_INFO: `/v3/roles` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:18:15.996839 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f01c5a-7182-4af0-b363-d88c2dbb421d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:18:15.998448 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f01c5a-7182-4af0-b363-d88c2dbb421d admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:18:16.010371 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1078/2155] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Aug 24 06:18:15 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 06:18:16.014929 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc6e96b-02a1-40f8-8782-dc5cf0b4bc8f admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:18:16.027669 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc6e96b-02a1-40f8-8782-dc5cf0b4bc8f admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:18:16.032603 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0861c65e-0684-4ad8-b368-5ee2a58ab632 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:18:16.033263 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0861c65e-0684-4ad8-b368-5ee2a58ab632 admin admin] REQUEST_METHOD: `PUT` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:18:16.033639 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0861c65e-0684-4ad8-b368-5ee2a58ab632 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:18:16.033923 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0861c65e-0684-4ad8-b368-5ee2a58ab632 admin admin] PATH_INFO: `/v3/projects/f1a762f10d5342c4a34affa811d060ee/users/995c6173024d4a02ab322e1cd1e4a742/roles/49e47cd157d743f6905f1db07ec878a7` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:18:16.035843 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0861c65e-0684-4ad8-b368-5ee2a58ab632 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f1a762f10d5342c4a34affa811d060ee, user_id=995c6173024d4a02ab322e1cd1e4a742, role_id=49e47cd157d743f6905f1db07ec878a7)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:18:16.038612 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0861c65e-0684-4ad8-b368-5ee2a58ab632 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:18:16.050126 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1078/2156] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Aug 24 06:18:16 2022] PUT /identity/v3/projects/f1a762f10d5342c4a34affa811d060ee/users/995c6173024d4a02ab322e1cd1e4a742/roles/49e47cd157d743f6905f1db07ec878a7 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:18:16.061751 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5123df1f-8b88-4078-86d1-c5af4973092d None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:18:16.062119 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5123df1f-8b88-4078-86d1-c5af4973092d None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:18:16.063081 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5123df1f-8b88-4078-86d1-c5af4973092d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:18:16.092336 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-5123df1f-8b88-4078-86d1-c5af4973092d None None] MFA Rules not processed for user `995c6173024d4a02ab322e1cd1e4a742`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:18:16.093646 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-5123df1f-8b88-4078-86d1-c5af4973092d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:18:16.173907 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1079/2157] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:18:16 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:18:16.186093 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0861c65e-0684-4ad8-b368-5ee2a58ab632 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:18:16.188428 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-0861c65e-0684-4ad8-b368-5ee2a58ab632 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:18:16.227861 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:18:16.253715 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0861c65e-0684-4ad8-b368-5ee2a58ab632 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:18:16.257379 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b9b645-4d45-44e0-b794-90980b949bea service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c4def55ac0e48b29836bc5c64bc7402', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:18:16.257975 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b9b645-4d45-44e0-b794-90980b949bea service cinder] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:18:16.258251 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b9b645-4d45-44e0-b794-90980b949bea service cinder] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:18:16.258517 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b9b645-4d45-44e0-b794-90980b949bea service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:18:16.334986 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b9b645-4d45-44e0-b794-90980b949bea service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:18:16.336694 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b9b645-4d45-44e0-b794-90980b949bea service cinder] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:18:16.347675 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1079/2158] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:18:16 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:18:18.675994 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5123df1f-8b88-4078-86d1-c5af4973092d None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:18:18.677531 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-5123df1f-8b88-4078-86d1-c5af4973092d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:18:18.691323 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5123df1f-8b88-4078-86d1-c5af4973092d None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:18:18.695854 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48a38cc-d13a-44f6-b31b-0928d8be8855 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c4def55ac0e48b29836bc5c64bc7402', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:18:18.696700 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48a38cc-d13a-44f6-b31b-0928d8be8855 service cinder] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:18:18.697126 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48a38cc-d13a-44f6-b31b-0928d8be8855 service cinder] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:18:18.697543 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48a38cc-d13a-44f6-b31b-0928d8be8855 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:18:18.778816 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48a38cc-d13a-44f6-b31b-0928d8be8855 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:18:18.778816 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48a38cc-d13a-44f6-b31b-0928d8be8855 service cinder] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:18:18.801805 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1080/2159] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:18:18 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:18:19.504920 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01776f9b-add6-47d5-9042-07f588d65fc3 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:18:19.505420 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01776f9b-add6-47d5-9042-07f588d65fc3 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:18:19.505420 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01776f9b-add6-47d5-9042-07f588d65fc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:18:19.535744 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-01776f9b-add6-47d5-9042-07f588d65fc3 None None] MFA Rules not processed for user `069dc6f771a74dd4ab22d9d5e68e3cc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:18:19.537021 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-01776f9b-add6-47d5-9042-07f588d65fc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:18:19.629526 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1080/2160] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:18:19 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 06:18:19.640823 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2ac6b1-9d32-4695-bec6-c2d93c588af0 None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:18:19.641106 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2ac6b1-9d32-4695-bec6-c2d93c588af0 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:18:19.641381 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2ac6b1-9d32-4695-bec6-c2d93c588af0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:18:19.662264 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-6e2ac6b1-9d32-4695-bec6-c2d93c588af0 None None] MFA Rules not processed for user `069dc6f771a74dd4ab22d9d5e68e3cc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:18:19.663484 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-6e2ac6b1-9d32-4695-bec6-c2d93c588af0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:18:19.670775 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1081/2161] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:18:19 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:18:19.683474 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01776f9b-add6-47d5-9042-07f588d65fc3 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:18:19.686327 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-01776f9b-add6-47d5-9042-07f588d65fc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:18:19.783274 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01776f9b-add6-47d5-9042-07f588d65fc3 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:18:19.792261 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab24dcd7-da7f-4dab-b4e7-5946d77094cb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb6d3743340241f98cacb9e0d063298c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:18:19.793180 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab24dcd7-da7f-4dab-b4e7-5946d77094cb service nova] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:18:19.793530 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab24dcd7-da7f-4dab-b4e7-5946d77094cb service nova] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:18:19.793921 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab24dcd7-da7f-4dab-b4e7-5946d77094cb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:18:19.805840 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab24dcd7-da7f-4dab-b4e7-5946d77094cb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:18:19.808768 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab24dcd7-da7f-4dab-b4e7-5946d77094cb service nova] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:18:19.821565 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1081/2162] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:18:19 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:18:20.626791 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2ac6b1-9d32-4695-bec6-c2d93c588af0 None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:18:20.642110 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2ac6b1-9d32-4695-bec6-c2d93c588af0 None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:18:20.643987 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c721772-b99b-499e-8216-2ac92cbb8bf5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c4def55ac0e48b29836bc5c64bc7402', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:18:20.644861 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c721772-b99b-499e-8216-2ac92cbb8bf5 service cinder] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:18:20.645325 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c721772-b99b-499e-8216-2ac92cbb8bf5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:18:20.645774 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c721772-b99b-499e-8216-2ac92cbb8bf5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:18:20.662494 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c721772-b99b-499e-8216-2ac92cbb8bf5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:18:20.664034 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c721772-b99b-499e-8216-2ac92cbb8bf5 service cinder] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:18:20.700186 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1082/2163] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:18:20 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 74 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 06:18:21.225630 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a9cbc9-88de-4a08-983e-2c029c2cf083 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:18:21.226518 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a9cbc9-88de-4a08-983e-2c029c2cf083 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:18:21.226796 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a9cbc9-88de-4a08-983e-2c029c2cf083 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:18:21.246448 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:18:21.268260 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-71a9cbc9-88de-4a08-983e-2c029c2cf083 None None] MFA Rules not processed for user `069dc6f771a74dd4ab22d9d5e68e3cc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:18:21.269915 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-71a9cbc9-88de-4a08-983e-2c029c2cf083 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:18:21.286681 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1082/2164] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:18:21 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 06:18:24.065317 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d4941a-adc7-4709-b388-2cc1f1ade9ab None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:18:24.066055 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d4941a-adc7-4709-b388-2cc1f1ade9ab None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:18:24.066208 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d4941a-adc7-4709-b388-2cc1f1ade9ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:18:24.075576 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:18:24.088547 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-68d4941a-adc7-4709-b388-2cc1f1ade9ab None None] MFA Rules not processed for user `b82726a12436435cb931b986ef884c2e`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:18:24.090146 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-68d4941a-adc7-4709-b388-2cc1f1ade9ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:18:24.158472 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1083/2165] 10.222.0.62 () {58 vars in 1009 bytes} [Wed Aug 24 06:18:24 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 06:18:24.170610 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a9cbc9-88de-4a08-983e-2c029c2cf083 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:18:24.176622 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-71a9cbc9-88de-4a08-983e-2c029c2cf083 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:18:24.279465 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a9cbc9-88de-4a08-983e-2c029c2cf083 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:18:24.285700 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73c6264-90f3-4038-a685-25bf186bf6e8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14d4fed04fb144d08918bd5d4f808d23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:18:24.286637 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73c6264-90f3-4038-a685-25bf186bf6e8 service swift] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:18:24.287103 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73c6264-90f3-4038-a685-25bf186bf6e8 service swift] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:18:24.287728 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73c6264-90f3-4038-a685-25bf186bf6e8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:18:24.301401 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73c6264-90f3-4038-a685-25bf186bf6e8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:18:24.304660 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73c6264-90f3-4038-a685-25bf186bf6e8 service swift] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:18:24.316824 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1083/2166] 10.222.0.62 () {60 vars in 1384 bytes} [Wed Aug 24 06:18:24 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 06:18:25.673656 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16ea4d0-1d23-4dd8-a3d0-a05d98537e90 None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:18:25.673656 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16ea4d0-1d23-4dd8-a3d0-a05d98537e90 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:18:25.673656 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16ea4d0-1d23-4dd8-a3d0-a05d98537e90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:18:25.695862 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-a16ea4d0-1d23-4dd8-a3d0-a05d98537e90 None None] MFA Rules not processed for user `b82726a12436435cb931b986ef884c2e`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:18:25.701829 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-a16ea4d0-1d23-4dd8-a3d0-a05d98537e90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:18:25.705216 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1084/2167] 10.222.0.62 () {58 vars in 967 bytes} [Wed Aug 24 06:18:25 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:18:25.720670 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73c6264-90f3-4038-a685-25bf186bf6e8 service swift] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:18:25.731041 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73c6264-90f3-4038-a685-25bf186bf6e8 service swift] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:18:25.736180 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72d64a2-b05b-4129-94b9-56e45c1b3869 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14d4fed04fb144d08918bd5d4f808d23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:18:25.736809 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72d64a2-b05b-4129-94b9-56e45c1b3869 service swift] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:18:25.737113 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72d64a2-b05b-4129-94b9-56e45c1b3869 service swift] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:18:25.737388 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72d64a2-b05b-4129-94b9-56e45c1b3869 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:18:25.746363 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72d64a2-b05b-4129-94b9-56e45c1b3869 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:18:25.748465 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72d64a2-b05b-4129-94b9-56e45c1b3869 service swift] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:18:25.757725 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1084/2168] 10.222.0.62 () {60 vars in 1384 bytes} [Wed Aug 24 06:18:25 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 06:18:26.482913 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ba447f-6b70-41bd-9c00-3d95c9ec4e5e None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:18:26.483195 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ba447f-6b70-41bd-9c00-3d95c9ec4e5e None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:18:26.483477 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ba447f-6b70-41bd-9c00-3d95c9ec4e5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:18:26.505049 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-85ba447f-6b70-41bd-9c00-3d95c9ec4e5e None None] MFA Rules not processed for user `b82726a12436435cb931b986ef884c2e`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:18:26.506249 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-85ba447f-6b70-41bd-9c00-3d95c9ec4e5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:18:26.513503 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1085/2169] 10.222.0.62 () {58 vars in 1009 bytes} [Wed Aug 24 06:18:26 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:18:26.525198 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72d64a2-b05b-4129-94b9-56e45c1b3869 service swift] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:18:26.535466 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72d64a2-b05b-4129-94b9-56e45c1b3869 service swift] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:18:26.540421 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c79fea-15c8-48ed-9157-57dfa2483415 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14d4fed04fb144d08918bd5d4f808d23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:18:26.541032 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c79fea-15c8-48ed-9157-57dfa2483415 service swift] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:18:26.541306 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c79fea-15c8-48ed-9157-57dfa2483415 service swift] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:18:26.541571 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c79fea-15c8-48ed-9157-57dfa2483415 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:18:26.550212 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c79fea-15c8-48ed-9157-57dfa2483415 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:18:26.551826 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c79fea-15c8-48ed-9157-57dfa2483415 service swift] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:18:26.559687 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1085/2170] 10.222.0.62 () {60 vars in 1384 bytes} [Wed Aug 24 06:18:26 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 24 06:18:27.121699 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afd60ca-3a6b-4875-9e6e-3cf793dc9b38 None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:18:27.122168 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afd60ca-3a6b-4875-9e6e-3cf793dc9b38 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:18:27.122335 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afd60ca-3a6b-4875-9e6e-3cf793dc9b38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:18:27.141993 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-7afd60ca-3a6b-4875-9e6e-3cf793dc9b38 None None] MFA Rules not processed for user `b82726a12436435cb931b986ef884c2e`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:18:27.143171 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-7afd60ca-3a6b-4875-9e6e-3cf793dc9b38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:18:27.151014 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1086/2171] 10.222.0.62 () {58 vars in 967 bytes} [Wed Aug 24 06:18:27 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:18:27.156136 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c79fea-15c8-48ed-9157-57dfa2483415 service swift] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:18:27.165020 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afd60ca-3a6b-4875-9e6e-3cf793dc9b38 None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:18:27.188720 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afd60ca-3a6b-4875-9e6e-3cf793dc9b38 None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:18:27.194212 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb62be0-7a3f-4da2-8ff7-0c5d70f63b0f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14d4fed04fb144d08918bd5d4f808d23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:18:27.195274 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb62be0-7a3f-4da2-8ff7-0c5d70f63b0f service swift] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:18:27.195710 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb62be0-7a3f-4da2-8ff7-0c5d70f63b0f service swift] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:18:27.195987 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb62be0-7a3f-4da2-8ff7-0c5d70f63b0f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:18:27.205045 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb62be0-7a3f-4da2-8ff7-0c5d70f63b0f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:18:27.206616 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb62be0-7a3f-4da2-8ff7-0c5d70f63b0f service swift] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:18:27.215091 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1087/2172] 10.222.0.62 () {60 vars in 1384 bytes} [Wed Aug 24 06:18:27 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 06:18:27.233710 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c79fea-15c8-48ed-9157-57dfa2483415 service swift] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:18:27.242493 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53367e69-355f-473a-85af-c4273ec01df0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:18:27.243534 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53367e69-355f-473a-85af-c4273ec01df0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:18:27.243843 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53367e69-355f-473a-85af-c4273ec01df0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:18:27.244115 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53367e69-355f-473a-85af-c4273ec01df0 admin admin] PATH_INFO: `/v3/users/e81a5462bd354f25bc4d237057264729` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:18:27.245374 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53367e69-355f-473a-85af-c4273ec01df0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e81a5462bd354f25bc4d237057264729)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:18:27.247326 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53367e69-355f-473a-85af-c4273ec01df0 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:18:27.453471 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-53367e69-355f-473a-85af-c4273ec01df0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e81a5462bd354f25bc4d237057264729'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:18:27.463985 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-53367e69-355f-473a-85af-c4273ec01df0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e81a5462bd354f25bc4d237057264729'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:18:27.464775 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-53367e69-355f-473a-85af-c4273ec01df0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e81a5462bd354f25bc4d237057264729'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:18:27.473237 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:18:27.524120 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-53367e69-355f-473a-85af-c4273ec01df0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e81a5462bd354f25bc4d237057264729'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:18:27.537471 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-53367e69-355f-473a-85af-c4273ec01df0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e81a5462bd354f25bc4d237057264729'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:18:27.543530 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1086/2173] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Aug 24 06:18:27 2022] DELETE /identity/v3/users/e81a5462bd354f25bc4d237057264729 => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:18:27.548027 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc49966-6a93-442b-a41e-df5247028eac None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:18:27.548309 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc49966-6a93-442b-a41e-df5247028eac None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:18:27.548616 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc49966-6a93-442b-a41e-df5247028eac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:18:27.567791 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-4dc49966-6a93-442b-a41e-df5247028eac None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:18:27.569071 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-4dc49966-6a93-442b-a41e-df5247028eac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:18:27.630909 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1088/2174] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:18:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:18:27.640669 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53367e69-355f-473a-85af-c4273ec01df0 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:18:27.642113 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-53367e69-355f-473a-85af-c4273ec01df0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:18:27.715689 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53367e69-355f-473a-85af-c4273ec01df0 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:18:27.719277 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc982bd9-56df-4cdc-8ad6-fc2a81e8bdd6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bb6c79db75e4d169331c02133d1ec32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:18:27.719881 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc982bd9-56df-4cdc-8ad6-fc2a81e8bdd6 service neutron] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:18:27.720154 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc982bd9-56df-4cdc-8ad6-fc2a81e8bdd6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:18:27.720471 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc982bd9-56df-4cdc-8ad6-fc2a81e8bdd6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:18:27.732811 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc982bd9-56df-4cdc-8ad6-fc2a81e8bdd6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:18:27.734301 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc982bd9-56df-4cdc-8ad6-fc2a81e8bdd6 service neutron] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:18:27.744269 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1087/2175] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:18:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:18:28.239503 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc49966-6a93-442b-a41e-df5247028eac None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:18:28.241436 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-4dc49966-6a93-442b-a41e-df5247028eac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:18:28.266089 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc49966-6a93-442b-a41e-df5247028eac None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:18:28.271641 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b72e05-3b4b-4c35-b5f9-7619640df0d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:18:28.272149 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b72e05-3b4b-4c35-b5f9-7619640df0d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:18:28.272434 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b72e05-3b4b-4c35-b5f9-7619640df0d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:18:28.272748 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b72e05-3b4b-4c35-b5f9-7619640df0d3 admin admin] PATH_INFO: `/v3/projects/ded654ad081f4766b5a287ad5c4a5758` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:18:28.273931 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b72e05-3b4b-4c35-b5f9-7619640df0d3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ded654ad081f4766b5a287ad5c4a5758)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:18:28.275784 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b72e05-3b4b-4c35-b5f9-7619640df0d3 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:18:28.395734 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-d8b72e05-3b4b-4c35-b5f9-7619640df0d3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ded654ad081f4766b5a287ad5c4a5758', 'request_id': 'req-d8b72e05-3b4b-4c35-b5f9-7619640df0d3'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:18:28.402228 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-d8b72e05-3b4b-4c35-b5f9-7619640df0d3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ded654ad081f4766b5a287ad5c4a5758', 'request_id': 'req-d8b72e05-3b4b-4c35-b5f9-7619640df0d3'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:18:28.409890 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-d8b72e05-3b4b-4c35-b5f9-7619640df0d3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ded654ad081f4766b5a287ad5c4a5758', 'request_id': 'req-d8b72e05-3b4b-4c35-b5f9-7619640df0d3'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:18:28.467569 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:18:28.469601 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-d8b72e05-3b4b-4c35-b5f9-7619640df0d3 admin admin] The token cache is being invalidate because project ded654ad081f4766b5a287ad5c4a5758 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81851) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:18:28.469966 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-d8b72e05-3b4b-4c35-b5f9-7619640df0d3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:18:28.470676 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-d8b72e05-3b4b-4c35-b5f9-7619640df0d3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:18:28.472395 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1089/2176] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Aug 24 06:18:28 2022] DELETE /identity/v3/projects/ded654ad081f4766b5a287ad5c4a5758 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:18:28.476465 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc982bd9-56df-4cdc-8ad6-fc2a81e8bdd6 service neutron] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:18:28.488354 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc982bd9-56df-4cdc-8ad6-fc2a81e8bdd6 service neutron] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:18:28.551155 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5026192-40ef-465b-9088-8c1bae13cbab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:18:28.551786 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5026192-40ef-465b-9088-8c1bae13cbab admin admin] REQUEST_METHOD: `DELETE` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:18:28.552066 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5026192-40ef-465b-9088-8c1bae13cbab admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:18:28.552339 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5026192-40ef-465b-9088-8c1bae13cbab admin admin] PATH_INFO: `/v3/users/a0443372b1c14d8085c531484e4c516c` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:18:28.553494 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5026192-40ef-465b-9088-8c1bae13cbab admin admin] RBAC: Authorizing `identity:delete_user(user_id=a0443372b1c14d8085c531484e4c516c)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:18:28.555405 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5026192-40ef-465b-9088-8c1bae13cbab admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:18:29.067577 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-f5026192-40ef-465b-9088-8c1bae13cbab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a0443372b1c14d8085c531484e4c516c'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:18:29.067577 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-f5026192-40ef-465b-9088-8c1bae13cbab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a0443372b1c14d8085c531484e4c516c'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:18:29.067577 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-f5026192-40ef-465b-9088-8c1bae13cbab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a0443372b1c14d8085c531484e4c516c'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:18:29.667661 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-f5026192-40ef-465b-9088-8c1bae13cbab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a0443372b1c14d8085c531484e4c516c'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:18:29.681792 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-f5026192-40ef-465b-9088-8c1bae13cbab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a0443372b1c14d8085c531484e4c516c'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:18:29.684208 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:18:29.689112 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1088/2177] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Aug 24 06:18:28 2022] DELETE /identity/v3/users/a0443372b1c14d8085c531484e4c516c => generated 0 bytes in 1213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:18:30.263159 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b72e05-3b4b-4c35-b5f9-7619640df0d3 admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:18:30.266688 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-d8b72e05-3b4b-4c35-b5f9-7619640df0d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:18:30.282195 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b72e05-3b4b-4c35-b5f9-7619640df0d3 admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:18:30.343840 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef2cfbc-b845-4863-88bb-60a3160c3323 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:18:30.344471 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef2cfbc-b845-4863-88bb-60a3160c3323 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:18:30.344780 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef2cfbc-b845-4863-88bb-60a3160c3323 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:18:30.345083 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef2cfbc-b845-4863-88bb-60a3160c3323 admin admin] PATH_INFO: `/v3/projects/6a4243fe591b4947933d358531a20c55` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:18:30.346192 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef2cfbc-b845-4863-88bb-60a3160c3323 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6a4243fe591b4947933d358531a20c55)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:18:30.348087 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef2cfbc-b845-4863-88bb-60a3160c3323 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:18:30.467492 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-0ef2cfbc-b845-4863-88bb-60a3160c3323 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6a4243fe591b4947933d358531a20c55', 'request_id': 'req-0ef2cfbc-b845-4863-88bb-60a3160c3323'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:18:30.472997 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-0ef2cfbc-b845-4863-88bb-60a3160c3323 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6a4243fe591b4947933d358531a20c55', 'request_id': 'req-0ef2cfbc-b845-4863-88bb-60a3160c3323'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:18:30.479113 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-0ef2cfbc-b845-4863-88bb-60a3160c3323 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6a4243fe591b4947933d358531a20c55', 'request_id': 'req-0ef2cfbc-b845-4863-88bb-60a3160c3323'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:18:30.546453 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-0ef2cfbc-b845-4863-88bb-60a3160c3323 admin admin] The token cache is being invalidate because project 6a4243fe591b4947933d358531a20c55 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81851) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:18:30.546743 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-0ef2cfbc-b845-4863-88bb-60a3160c3323 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:18:30.547366 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-0ef2cfbc-b845-4863-88bb-60a3160c3323 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:18:30.548889 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1090/2178] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Aug 24 06:18:30 2022] DELETE /identity/v3/projects/6a4243fe591b4947933d358531a20c55 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:18:30.553982 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5026192-40ef-465b-9088-8c1bae13cbab admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:18:30.555470 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-f5026192-40ef-465b-9088-8c1bae13cbab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:18:30.568163 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5026192-40ef-465b-9088-8c1bae13cbab admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:18:30.622923 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7932ae-4971-4da5-bbbe-478aa022579f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:18:30.623675 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7932ae-4971-4da5-bbbe-478aa022579f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:18:30.623841 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7932ae-4971-4da5-bbbe-478aa022579f admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:18:30.624111 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7932ae-4971-4da5-bbbe-478aa022579f admin admin] PATH_INFO: `/v3/users/995c6173024d4a02ab322e1cd1e4a742` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:18:30.625466 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7932ae-4971-4da5-bbbe-478aa022579f admin admin] RBAC: Authorizing `identity:delete_user(user_id=995c6173024d4a02ab322e1cd1e4a742)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:18:30.627498 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7932ae-4971-4da5-bbbe-478aa022579f admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:18:30.777316 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-ab7932ae-4971-4da5-bbbe-478aa022579f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '995c6173024d4a02ab322e1cd1e4a742'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:18:30.784858 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-ab7932ae-4971-4da5-bbbe-478aa022579f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '995c6173024d4a02ab322e1cd1e4a742'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:18:30.785546 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-ab7932ae-4971-4da5-bbbe-478aa022579f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '995c6173024d4a02ab322e1cd1e4a742'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:18:30.855678 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-ab7932ae-4971-4da5-bbbe-478aa022579f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '995c6173024d4a02ab322e1cd1e4a742'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:18:30.868668 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-ab7932ae-4971-4da5-bbbe-478aa022579f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '995c6173024d4a02ab322e1cd1e4a742'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:18:30.874569 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1089/2179] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Aug 24 06:18:30 2022] DELETE /identity/v3/users/995c6173024d4a02ab322e1cd1e4a742 => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:18:31.285602 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef2cfbc-b845-4863-88bb-60a3160c3323 admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:18:31.287260 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-0ef2cfbc-b845-4863-88bb-60a3160c3323 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:18:31.301666 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef2cfbc-b845-4863-88bb-60a3160c3323 admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:18:31.360732 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a48dd2b-637d-4f35-a15a-f2bd302f1edb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:18:31.361366 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a48dd2b-637d-4f35-a15a-f2bd302f1edb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:18:31.361646 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a48dd2b-637d-4f35-a15a-f2bd302f1edb admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:18:31.361919 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a48dd2b-637d-4f35-a15a-f2bd302f1edb admin admin] PATH_INFO: `/v3/projects/f1a762f10d5342c4a34affa811d060ee` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:18:31.363390 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a48dd2b-637d-4f35-a15a-f2bd302f1edb admin admin] RBAC: Authorizing `identity:delete_project(project_id=f1a762f10d5342c4a34affa811d060ee)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:18:31.365176 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a48dd2b-637d-4f35-a15a-f2bd302f1edb admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:18:31.472917 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:18:31.484832 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-5a48dd2b-637d-4f35-a15a-f2bd302f1edb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f1a762f10d5342c4a34affa811d060ee', 'request_id': 'req-5a48dd2b-637d-4f35-a15a-f2bd302f1edb'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:18:31.490467 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-5a48dd2b-637d-4f35-a15a-f2bd302f1edb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f1a762f10d5342c4a34affa811d060ee', 'request_id': 'req-5a48dd2b-637d-4f35-a15a-f2bd302f1edb'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:18:31.497042 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-5a48dd2b-637d-4f35-a15a-f2bd302f1edb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f1a762f10d5342c4a34affa811d060ee', 'request_id': 'req-5a48dd2b-637d-4f35-a15a-f2bd302f1edb'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:18:31.557769 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-5a48dd2b-637d-4f35-a15a-f2bd302f1edb admin admin] The token cache is being invalidate because project f1a762f10d5342c4a34affa811d060ee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81851) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:18:31.558061 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-5a48dd2b-637d-4f35-a15a-f2bd302f1edb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:18:31.558692 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-5a48dd2b-637d-4f35-a15a-f2bd302f1edb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:18:31.560225 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1091/2180] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Aug 24 06:18:31 2022] DELETE /identity/v3/projects/f1a762f10d5342c4a34affa811d060ee => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:18:31.565584 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311c555f-5740-41d6-8655-f88000dc59c7 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:18:31.565867 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311c555f-5740-41d6-8655-f88000dc59c7 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:18:31.566139 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311c555f-5740-41d6-8655-f88000dc59c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:18:31.585985 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-311c555f-5740-41d6-8655-f88000dc59c7 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:18:31.587177 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-311c555f-5740-41d6-8655-f88000dc59c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:18:31.646589 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1090/2181] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:18:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:18:31.654247 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3aff6d-5b4f-476e-b948-e483b9ccd047 None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:18:31.654538 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3aff6d-5b4f-476e-b948-e483b9ccd047 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:18:31.654819 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3aff6d-5b4f-476e-b948-e483b9ccd047 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:18:31.675340 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-bf3aff6d-5b4f-476e-b948-e483b9ccd047 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:18:31.676611 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-bf3aff6d-5b4f-476e-b948-e483b9ccd047 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:18:31.684483 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1092/2182] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:18:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:18:31.688734 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311c555f-5740-41d6-8655-f88000dc59c7 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:18:31.699924 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311c555f-5740-41d6-8655-f88000dc59c7 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:18:31.704467 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75519df-8442-470c-82e4-45aeaec09631 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:18:31.705091 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75519df-8442-470c-82e4-45aeaec09631 admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:18:31.705366 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75519df-8442-470c-82e4-45aeaec09631 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:18:31.705640 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75519df-8442-470c-82e4-45aeaec09631 admin admin] PATH_INFO: `/v3/domains` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:18:31.706276 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75519df-8442-470c-82e4-45aeaec09631 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:18:31.706576 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75519df-8442-470c-82e4-45aeaec09631 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:18:31.708186 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75519df-8442-470c-82e4-45aeaec09631 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:18:31.719851 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1091/2183] 10.222.0.62 () {58 vars in 1153 bytes} [Wed Aug 24 06:18:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:18:31.724112 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3aff6d-5b4f-476e-b948-e483b9ccd047 None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:18:31.733608 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3aff6d-5b4f-476e-b948-e483b9ccd047 None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:18:31.738985 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4596c5ed-521a-4a54-933c-e94ac8b947bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:18:31.739578 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4596c5ed-521a-4a54-933c-e94ac8b947bf admin admin] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:18:31.739859 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4596c5ed-521a-4a54-933c-e94ac8b947bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:18:31.740133 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4596c5ed-521a-4a54-933c-e94ac8b947bf admin admin] PATH_INFO: `/v3/projects` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:18:31.740830 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4596c5ed-521a-4a54-933c-e94ac8b947bf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:18:31.742767 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4596c5ed-521a-4a54-933c-e94ac8b947bf admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:18:31.810744 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1093/2184] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Aug 24 06:18:31 2022] POST /identity/v3/projects => generated 381 bytes in 87 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:18:31.815553 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75519df-8442-470c-82e4-45aeaec09631 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:18:31.825628 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75519df-8442-470c-82e4-45aeaec09631 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:18:31.829281 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:18:31.881851 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b633434e-a844-4f95-9ec9-672c84e9c5fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:18:31.882525 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b633434e-a844-4f95-9ec9-672c84e9c5fa admin admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:18:31.882685 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b633434e-a844-4f95-9ec9-672c84e9c5fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:18:31.882953 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b633434e-a844-4f95-9ec9-672c84e9c5fa admin admin] PATH_INFO: `/v3/users` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:18:31.883558 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b633434e-a844-4f95-9ec9-672c84e9c5fa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:18:31.885509 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b633434e-a844-4f95-9ec9-672c84e9c5fa admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:18:31.952370 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1092/2185] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Aug 24 06:18:31 2022] POST /identity/v3/users => generated 291 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 06:18:31.956744 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4596c5ed-521a-4a54-933c-e94ac8b947bf admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:18:31.967343 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4596c5ed-521a-4a54-933c-e94ac8b947bf admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:18:31.972079 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec438ab-f919-4640-a92d-0e9573f6dbad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:18:31.972711 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec438ab-f919-4640-a92d-0e9573f6dbad admin admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:18:31.972991 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec438ab-f919-4640-a92d-0e9573f6dbad admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:18:31.973265 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec438ab-f919-4640-a92d-0e9573f6dbad admin admin] PATH_INFO: `/v3/roles` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:18:31.973899 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec438ab-f919-4640-a92d-0e9573f6dbad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:18:31.975486 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec438ab-f919-4640-a92d-0e9573f6dbad admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:18:31.985463 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1094/2186] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Aug 24 06:18:31 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:18:31.990047 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b633434e-a844-4f95-9ec9-672c84e9c5fa admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:18:31.999813 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b633434e-a844-4f95-9ec9-672c84e9c5fa admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:18:32.004919 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8006e194-e52d-41da-911e-c31ca906d836 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:18:32.006889 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8006e194-e52d-41da-911e-c31ca906d836 admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:18:32.006889 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8006e194-e52d-41da-911e-c31ca906d836 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:18:32.006889 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8006e194-e52d-41da-911e-c31ca906d836 admin admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:18:32.006889 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8006e194-e52d-41da-911e-c31ca906d836 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:18:32.008200 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8006e194-e52d-41da-911e-c31ca906d836 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:18:32.018801 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1093/2187] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Aug 24 06:18:31 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:18:32.022873 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec438ab-f919-4640-a92d-0e9573f6dbad admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:18:32.032902 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec438ab-f919-4640-a92d-0e9573f6dbad admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:18:32.038088 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1d4faf-9fed-4b60-96be-bb1c36b9d852 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:18:32.038727 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1d4faf-9fed-4b60-96be-bb1c36b9d852 admin admin] REQUEST_METHOD: `PUT` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:18:32.039004 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1d4faf-9fed-4b60-96be-bb1c36b9d852 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:18:32.039275 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1d4faf-9fed-4b60-96be-bb1c36b9d852 admin admin] PATH_INFO: `/v3/projects/7315c7714a7f40ed852dde74d497d231/users/4cb18bdb6d404cd5b303ce0a29212e4f/roles/580aa7dbb38d4cff8f9d1bb8983f99ef` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:18:32.054519 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1d4faf-9fed-4b60-96be-bb1c36b9d852 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7315c7714a7f40ed852dde74d497d231, user_id=4cb18bdb6d404cd5b303ce0a29212e4f, role_id=580aa7dbb38d4cff8f9d1bb8983f99ef)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:18:32.057265 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1d4faf-9fed-4b60-96be-bb1c36b9d852 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:18:32.141296 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1095/2188] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Aug 24 06:18:32 2022] PUT /identity/v3/projects/7315c7714a7f40ed852dde74d497d231/users/4cb18bdb6d404cd5b303ce0a29212e4f/roles/580aa7dbb38d4cff8f9d1bb8983f99ef => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:18:32.146461 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20777016-62de-483f-ad1f-7eff938afaad None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:18:32.150320 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20777016-62de-483f-ad1f-7eff938afaad None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:18:32.150320 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20777016-62de-483f-ad1f-7eff938afaad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:18:32.166661 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-20777016-62de-483f-ad1f-7eff938afaad None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:18:32.167899 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-20777016-62de-483f-ad1f-7eff938afaad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:18:32.231742 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1094/2189] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:18:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:18:32.242300 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1d4faf-9fed-4b60-96be-bb1c36b9d852 admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:18:32.244021 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-2f1d4faf-9fed-4b60-96be-bb1c36b9d852 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:18:32.300665 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1d4faf-9fed-4b60-96be-bb1c36b9d852 admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:18:32.304261 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca24a05e-ea83-4364-a759-d798a090cb09 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bb6c79db75e4d169331c02133d1ec32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:18:32.304889 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca24a05e-ea83-4364-a759-d798a090cb09 service neutron] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:18:32.305166 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca24a05e-ea83-4364-a759-d798a090cb09 service neutron] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:18:32.305434 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca24a05e-ea83-4364-a759-d798a090cb09 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:18:32.314066 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca24a05e-ea83-4364-a759-d798a090cb09 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:18:32.315509 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca24a05e-ea83-4364-a759-d798a090cb09 service neutron] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:18:32.324613 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1096/2190] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:18:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:18:33.787318 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20777016-62de-483f-ad1f-7eff938afaad None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:18:33.791115 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-20777016-62de-483f-ad1f-7eff938afaad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:18:33.825526 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a08182-7802-4fca-a8e2-093e640cefb8 None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:18:33.825526 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a08182-7802-4fca-a8e2-093e640cefb8 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:18:33.825526 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a08182-7802-4fca-a8e2-093e640cefb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:18:33.826140 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20777016-62de-483f-ad1f-7eff938afaad None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:18:33.831008 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8396dcd6-fb42-4c87-869f-bf568a86c6f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:18:33.831522 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8396dcd6-fb42-4c87-869f-bf568a86c6f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:18:33.831962 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8396dcd6-fb42-4c87-869f-bf568a86c6f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:18:33.832442 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8396dcd6-fb42-4c87-869f-bf568a86c6f7 admin admin] PATH_INFO: `/v3/users/c9007aaaf32d4c22b3dc56c3cd1e08ed` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:18:33.834669 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8396dcd6-fb42-4c87-869f-bf568a86c6f7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c9007aaaf32d4c22b3dc56c3cd1e08ed)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:18:33.838171 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8396dcd6-fb42-4c87-869f-bf568a86c6f7 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:18:33.888804 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-32a08182-7802-4fca-a8e2-093e640cefb8 None None] MFA Rules not processed for user `4cb18bdb6d404cd5b303ce0a29212e4f`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:18:33.893272 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-32a08182-7802-4fca-a8e2-093e640cefb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:18:33.977740 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:18:34.073067 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1097/2191] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:18:33 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:18:34.111854 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-8396dcd6-fb42-4c87-869f-bf568a86c6f7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c9007aaaf32d4c22b3dc56c3cd1e08ed'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:18:34.122794 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-8396dcd6-fb42-4c87-869f-bf568a86c6f7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c9007aaaf32d4c22b3dc56c3cd1e08ed'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:18:34.126298 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-8396dcd6-fb42-4c87-869f-bf568a86c6f7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c9007aaaf32d4c22b3dc56c3cd1e08ed'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:18:34.198809 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-8396dcd6-fb42-4c87-869f-bf568a86c6f7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c9007aaaf32d4c22b3dc56c3cd1e08ed'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:18:34.213421 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a08182-7802-4fca-a8e2-093e640cefb8 None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:18:34.215138 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-32a08182-7802-4fca-a8e2-093e640cefb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:18:34.219397 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-8396dcd6-fb42-4c87-869f-bf568a86c6f7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c9007aaaf32d4c22b3dc56c3cd1e08ed'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:18:34.226195 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:18:34.245457 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1095/2192] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Aug 24 06:18:33 2022] DELETE /identity/v3/users/c9007aaaf32d4c22b3dc56c3cd1e08ed => generated 0 bytes in 455 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:18:34.247451 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8315ad34-c54f-40a2-ad13-15a7c271220e None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:18:34.247451 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8315ad34-c54f-40a2-ad13-15a7c271220e None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:18:34.247695 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8315ad34-c54f-40a2-ad13-15a7c271220e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:18:34.269746 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-8315ad34-c54f-40a2-ad13-15a7c271220e None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:18:34.271329 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-8315ad34-c54f-40a2-ad13-15a7c271220e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:18:34.315783 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a08182-7802-4fca-a8e2-093e640cefb8 None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:18:34.382471 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1096/2193] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:18:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:18:34.391540 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e13ca9-9540-4434-b0e8-3554ef5013c6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c4def55ac0e48b29836bc5c64bc7402', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:18:34.394025 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e13ca9-9540-4434-b0e8-3554ef5013c6 service cinder] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:18:34.394294 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e13ca9-9540-4434-b0e8-3554ef5013c6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:18:34.394957 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e13ca9-9540-4434-b0e8-3554ef5013c6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:18:34.397037 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-51e13ca9-9540-4434-b0e8-3554ef5013c6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:18:34.409411 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8315ad34-c54f-40a2-ad13-15a7c271220e None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:18:34.411686 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-8315ad34-c54f-40a2-ad13-15a7c271220e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:18:34.492813 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e13ca9-9540-4434-b0e8-3554ef5013c6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:18:34.496205 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e13ca9-9540-4434-b0e8-3554ef5013c6 service cinder] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:18:34.510393 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1098/2194] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:18:34 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 06:18:34.544964 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8315ad34-c54f-40a2-ad13-15a7c271220e None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:18:34.554905 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7252d004-a922-42a0-9b3b-b82871e77dfc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bb6c79db75e4d169331c02133d1ec32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:18:34.554905 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7252d004-a922-42a0-9b3b-b82871e77dfc service neutron] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:18:34.554905 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7252d004-a922-42a0-9b3b-b82871e77dfc service neutron] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:18:34.555680 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7252d004-a922-42a0-9b3b-b82871e77dfc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:18:34.576115 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7252d004-a922-42a0-9b3b-b82871e77dfc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:18:34.578397 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7252d004-a922-42a0-9b3b-b82871e77dfc service neutron] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:18:34.600292 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1097/2195] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:18:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:18:35.214635 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e13ca9-9540-4434-b0e8-3554ef5013c6 service cinder] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:18:35.216836 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-51e13ca9-9540-4434-b0e8-3554ef5013c6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:18:35.234997 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e13ca9-9540-4434-b0e8-3554ef5013c6 service cinder] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:18:35.240234 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99990a1-8693-465b-9885-71124f414d45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:18:35.240982 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99990a1-8693-465b-9885-71124f414d45 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:18:35.242582 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99990a1-8693-465b-9885-71124f414d45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:18:35.242582 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99990a1-8693-465b-9885-71124f414d45 admin admin] PATH_INFO: `/v3/projects/e2345e03d1a94b1ea2388246aeefedbb` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:18:35.243572 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99990a1-8693-465b-9885-71124f414d45 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e2345e03d1a94b1ea2388246aeefedbb)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:18:35.251483 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99990a1-8693-465b-9885-71124f414d45 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:18:35.461862 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-e99990a1-8693-465b-9885-71124f414d45 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e2345e03d1a94b1ea2388246aeefedbb', 'request_id': 'req-e99990a1-8693-465b-9885-71124f414d45'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:18:35.477721 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-e99990a1-8693-465b-9885-71124f414d45 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e2345e03d1a94b1ea2388246aeefedbb', 'request_id': 'req-e99990a1-8693-465b-9885-71124f414d45'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:18:35.496420 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-e99990a1-8693-465b-9885-71124f414d45 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e2345e03d1a94b1ea2388246aeefedbb', 'request_id': 'req-e99990a1-8693-465b-9885-71124f414d45'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:18:35.574818 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-e99990a1-8693-465b-9885-71124f414d45 admin admin] The token cache is being invalidate because project e2345e03d1a94b1ea2388246aeefedbb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81851) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:18:35.574818 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-e99990a1-8693-465b-9885-71124f414d45 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:18:35.574818 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-e99990a1-8693-465b-9885-71124f414d45 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:18:35.586275 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1099/2196] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Aug 24 06:18:35 2022] DELETE /identity/v3/projects/e2345e03d1a94b1ea2388246aeefedbb => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:18:35.799376 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7252d004-a922-42a0-9b3b-b82871e77dfc service neutron] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:18:35.804675 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-7252d004-a922-42a0-9b3b-b82871e77dfc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:18:35.812983 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7252d004-a922-42a0-9b3b-b82871e77dfc service neutron] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:18:35.872791 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d6902a-bc24-4cb5-9b33-251a45020af3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bb6c79db75e4d169331c02133d1ec32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:18:35.873447 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d6902a-bc24-4cb5-9b33-251a45020af3 service neutron] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:18:35.873729 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d6902a-bc24-4cb5-9b33-251a45020af3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:18:35.874017 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d6902a-bc24-4cb5-9b33-251a45020af3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:18:35.875787 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-65d6902a-bc24-4cb5-9b33-251a45020af3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:18:35.940385 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d6902a-bc24-4cb5-9b33-251a45020af3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:18:35.941851 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d6902a-bc24-4cb5-9b33-251a45020af3 service neutron] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:18:35.951719 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1098/2197] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:18:35 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:18:36.539533 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c61374-b901-485d-97fe-23420c06fd33 None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:18:36.540048 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c61374-b901-485d-97fe-23420c06fd33 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:18:36.540048 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c61374-b901-485d-97fe-23420c06fd33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:18:36.562571 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-48c61374-b901-485d-97fe-23420c06fd33 None None] MFA Rules not processed for user `4cb18bdb6d404cd5b303ce0a29212e4f`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:18:36.563444 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-48c61374-b901-485d-97fe-23420c06fd33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:18:36.637915 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1100/2198] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:18:36 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:18:36.719917 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:18:36.792766 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:18:39.577472 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d6902a-bc24-4cb5-9b33-251a45020af3 service neutron] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:18:39.579000 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-65d6902a-bc24-4cb5-9b33-251a45020af3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:18:39.645074 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d6902a-bc24-4cb5-9b33-251a45020af3 service neutron] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:18:39.649548 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3656cabf-c8e1-4892-87f0-ca8afb1468cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:18:39.650174 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3656cabf-c8e1-4892-87f0-ca8afb1468cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:18:39.650452 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3656cabf-c8e1-4892-87f0-ca8afb1468cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:18:39.650723 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3656cabf-c8e1-4892-87f0-ca8afb1468cc admin admin] PATH_INFO: `/v3/users/30c4582d2fa84c54aa9b42c4e40ba81d` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:18:39.651820 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3656cabf-c8e1-4892-87f0-ca8afb1468cc admin admin] RBAC: Authorizing `identity:delete_user(user_id=30c4582d2fa84c54aa9b42c4e40ba81d)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:18:39.653781 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3656cabf-c8e1-4892-87f0-ca8afb1468cc admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:18:39.937916 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-3656cabf-c8e1-4892-87f0-ca8afb1468cc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '30c4582d2fa84c54aa9b42c4e40ba81d'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:18:39.944719 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-3656cabf-c8e1-4892-87f0-ca8afb1468cc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '30c4582d2fa84c54aa9b42c4e40ba81d'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:18:39.945381 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-3656cabf-c8e1-4892-87f0-ca8afb1468cc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '30c4582d2fa84c54aa9b42c4e40ba81d'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:18:40.021470 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-3656cabf-c8e1-4892-87f0-ca8afb1468cc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '30c4582d2fa84c54aa9b42c4e40ba81d'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:18:40.037050 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-3656cabf-c8e1-4892-87f0-ca8afb1468cc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '30c4582d2fa84c54aa9b42c4e40ba81d'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:18:40.043376 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1099/2199] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Aug 24 06:18:39 2022] DELETE /identity/v3/users/30c4582d2fa84c54aa9b42c4e40ba81d => generated 0 bytes in 466 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:18:40.047826 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599f828d-d55f-4f0f-8b2f-79fb7f32dd18 None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:18:40.048366 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599f828d-d55f-4f0f-8b2f-79fb7f32dd18 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:18:40.048366 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599f828d-d55f-4f0f-8b2f-79fb7f32dd18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:18:40.067076 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-599f828d-d55f-4f0f-8b2f-79fb7f32dd18 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:18:40.073591 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-599f828d-d55f-4f0f-8b2f-79fb7f32dd18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:18:40.142823 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1101/2200] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:18:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:18:40.152365 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3656cabf-c8e1-4892-87f0-ca8afb1468cc admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:18:40.153938 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-3656cabf-c8e1-4892-87f0-ca8afb1468cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:18:40.218136 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3656cabf-c8e1-4892-87f0-ca8afb1468cc admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:18:40.221895 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce1a6e5-e8ef-4c09-b2bf-ffcb33ef8aa4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bb6c79db75e4d169331c02133d1ec32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:18:40.222495 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce1a6e5-e8ef-4c09-b2bf-ffcb33ef8aa4 service neutron] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:18:40.222768 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce1a6e5-e8ef-4c09-b2bf-ffcb33ef8aa4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:18:40.223032 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce1a6e5-e8ef-4c09-b2bf-ffcb33ef8aa4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:18:40.232841 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce1a6e5-e8ef-4c09-b2bf-ffcb33ef8aa4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:18:40.235246 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce1a6e5-e8ef-4c09-b2bf-ffcb33ef8aa4 service neutron] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:18:40.247579 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1100/2201] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:18:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:18:40.743884 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599f828d-d55f-4f0f-8b2f-79fb7f32dd18 None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:18:40.746943 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-599f828d-d55f-4f0f-8b2f-79fb7f32dd18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:18:40.812150 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599f828d-d55f-4f0f-8b2f-79fb7f32dd18 None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:18:40.829937 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74e96be-030d-4e0e-a459-ba66cb99c7bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:18:40.830588 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74e96be-030d-4e0e-a459-ba66cb99c7bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:18:40.830866 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74e96be-030d-4e0e-a459-ba66cb99c7bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:18:40.831164 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74e96be-030d-4e0e-a459-ba66cb99c7bf admin admin] PATH_INFO: `/v3/projects/bdc9217f36d54272b3b5db9a1150eb14` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:18:40.832517 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74e96be-030d-4e0e-a459-ba66cb99c7bf admin admin] RBAC: Authorizing `identity:delete_project(project_id=bdc9217f36d54272b3b5db9a1150eb14)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:18:40.834577 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74e96be-030d-4e0e-a459-ba66cb99c7bf admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:18:40.979133 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-f74e96be-030d-4e0e-a459-ba66cb99c7bf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bdc9217f36d54272b3b5db9a1150eb14', 'request_id': 'req-f74e96be-030d-4e0e-a459-ba66cb99c7bf'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:18:40.985716 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-f74e96be-030d-4e0e-a459-ba66cb99c7bf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bdc9217f36d54272b3b5db9a1150eb14', 'request_id': 'req-f74e96be-030d-4e0e-a459-ba66cb99c7bf'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:18:40.997846 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-f74e96be-030d-4e0e-a459-ba66cb99c7bf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bdc9217f36d54272b3b5db9a1150eb14', 'request_id': 'req-f74e96be-030d-4e0e-a459-ba66cb99c7bf'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:18:41.119676 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-f74e96be-030d-4e0e-a459-ba66cb99c7bf admin admin] The token cache is being invalidate because project bdc9217f36d54272b3b5db9a1150eb14 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81851) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:18:41.120831 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-f74e96be-030d-4e0e-a459-ba66cb99c7bf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:18:41.132895 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-f74e96be-030d-4e0e-a459-ba66cb99c7bf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:18:41.132895 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1102/2202] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Aug 24 06:18:40 2022] DELETE /identity/v3/projects/bdc9217f36d54272b3b5db9a1150eb14 => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:18:48.348450 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e803dee-bcdb-4597-b686-caa1d231175c None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:18:48.349005 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e803dee-bcdb-4597-b686-caa1d231175c None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:18:48.349128 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e803dee-bcdb-4597-b686-caa1d231175c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:18:48.368562 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-8e803dee-bcdb-4597-b686-caa1d231175c None None] MFA Rules not processed for user `b82726a12436435cb931b986ef884c2e`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:18:48.369763 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-8e803dee-bcdb-4597-b686-caa1d231175c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:18:48.380101 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:18:48.434006 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1101/2203] 10.222.0.62 () {58 vars in 1009 bytes} [Wed Aug 24 06:18:48 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:18:48.450098 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74e96be-030d-4e0e-a459-ba66cb99c7bf admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:18:48.451593 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-f74e96be-030d-4e0e-a459-ba66cb99c7bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:18:48.477895 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:DELETE=9,keystone:SELECT=89,keystone:INSERT=2 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:18:48.555263 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74e96be-030d-4e0e-a459-ba66cb99c7bf admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:18:48.560604 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262a1b9b-2d6f-46d5-94cd-c9c1ebd23535 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14d4fed04fb144d08918bd5d4f808d23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:18:48.561284 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262a1b9b-2d6f-46d5-94cd-c9c1ebd23535 service swift] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:18:48.562288 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262a1b9b-2d6f-46d5-94cd-c9c1ebd23535 service swift] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:18:48.562631 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262a1b9b-2d6f-46d5-94cd-c9c1ebd23535 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:18:48.573032 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262a1b9b-2d6f-46d5-94cd-c9c1ebd23535 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:18:48.576159 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262a1b9b-2d6f-46d5-94cd-c9c1ebd23535 service swift] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:18:48.592524 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1103/2204] 10.222.0.62 () {60 vars in 1384 bytes} [Wed Aug 24 06:18:48 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 06:18:54.731739 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a999faad-b8d8-46ee-9452-be89b6e845e8 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:18:54.732314 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a999faad-b8d8-46ee-9452-be89b6e845e8 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:18:54.732314 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a999faad-b8d8-46ee-9452-be89b6e845e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:18:54.780125 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-a999faad-b8d8-46ee-9452-be89b6e845e8 None None] MFA Rules not processed for user `6abfd0a28f9e482badae7018f1b7f9a1`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:18:54.781439 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-a999faad-b8d8-46ee-9452-be89b6e845e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:18:54.891360 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1102/2205] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:18:54 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:18:59.398393 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=27 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:19:04.884288 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=44 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:19:08.320574 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f128a2d-0c62-402f-bb12-911f66a0fec4 None None] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:19:08.321255 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f128a2d-0c62-402f-bb12-911f66a0fec4 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:19:08.321255 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f128a2d-0c62-402f-bb12-911f66a0fec4 None None] PATH_INFO: `/` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:19:08.322800 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1104/2206] 10.222.0.62 () {54 vars in 921 bytes} [Wed Aug 24 06:19:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:19:08.329009 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7620705f-942e-41ed-b60f-024b8cc603c4 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:19:08.329536 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7620705f-942e-41ed-b60f-024b8cc603c4 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:19:08.329536 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7620705f-942e-41ed-b60f-024b8cc603c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:19:08.349871 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-7620705f-942e-41ed-b60f-024b8cc603c4 None None] MFA Rules not processed for user `fb6d3743340241f98cacb9e0d063298c`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:19:08.351123 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-7620705f-942e-41ed-b60f-024b8cc603c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:19:08.420525 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1103/2207] 10.222.0.62 () {58 vars in 1018 bytes} [Wed Aug 24 06:19:08 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:19:08.432069 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f128a2d-0c62-402f-bb12-911f66a0fec4 None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:19:08.433770 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-7f128a2d-0c62-402f-bb12-911f66a0fec4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:19:08.496080 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f128a2d-0c62-402f-bb12-911f66a0fec4 None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:19:08.499763 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f936df5-0ae2-4bba-9ce9-ebd9ac4be9f1 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7d52162cbdc4218b577ecc1e69a88c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:19:08.504744 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f936df5-0ae2-4bba-9ce9-ebd9ac4be9f1 service glance] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:19:08.504744 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f936df5-0ae2-4bba-9ce9-ebd9ac4be9f1 service glance] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:19:08.504744 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f936df5-0ae2-4bba-9ce9-ebd9ac4be9f1 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:19:08.511352 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f936df5-0ae2-4bba-9ce9-ebd9ac4be9f1 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:19:08.513724 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f936df5-0ae2-4bba-9ce9-ebd9ac4be9f1 service glance] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:19:08.525443 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1105/2208] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:19:08 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:19:08.749654 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9a8b61-1839-4d03-8623-be1c3e8b23da None None] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:19:08.749654 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9a8b61-1839-4d03-8623-be1c3e8b23da None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:19:08.749654 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9a8b61-1839-4d03-8623-be1c3e8b23da None None] PATH_INFO: `/` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:19:08.750708 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1104/2209] 10.222.0.62 () {54 vars in 921 bytes} [Wed Aug 24 06:19:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:19:08.755509 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6969b81f-79da-43f8-8a89-32395f8f097d None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:19:08.755872 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6969b81f-79da-43f8-8a89-32395f8f097d None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:19:08.756366 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6969b81f-79da-43f8-8a89-32395f8f097d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:19:08.781040 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-6969b81f-79da-43f8-8a89-32395f8f097d None None] MFA Rules not processed for user `0bb6c79db75e4d169331c02133d1ec32`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:19:08.784240 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-6969b81f-79da-43f8-8a89-32395f8f097d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:19:08.836813 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1106/2210] 10.222.0.62 () {58 vars in 1018 bytes} [Wed Aug 24 06:19:08 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:19:08.849291 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9a8b61-1839-4d03-8623-be1c3e8b23da None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:19:08.851091 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-0f9a8b61-1839-4d03-8623-be1c3e8b23da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:19:08.866091 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9a8b61-1839-4d03-8623-be1c3e8b23da None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:19:08.870047 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1393dc85-cac7-4e6f-9cec-4858a0ec4fda service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bb6c79db75e4d169331c02133d1ec32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:19:08.870645 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1393dc85-cac7-4e6f-9cec-4858a0ec4fda service neutron] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:19:08.870963 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1393dc85-cac7-4e6f-9cec-4858a0ec4fda service neutron] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:19:08.871239 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1393dc85-cac7-4e6f-9cec-4858a0ec4fda service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:19:08.878554 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1393dc85-cac7-4e6f-9cec-4858a0ec4fda service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:19:08.880028 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1393dc85-cac7-4e6f-9cec-4858a0ec4fda service neutron] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:19:08.887963 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1105/2211] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:19:08 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:19:08.972185 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab43eda-bf1d-4b3e-b010-f3bf778e74f5 None None] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:19:08.972471 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab43eda-bf1d-4b3e-b010-f3bf778e74f5 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:19:08.972783 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab43eda-bf1d-4b3e-b010-f3bf778e74f5 None None] PATH_INFO: `/` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:19:08.974146 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1107/2212] 10.222.0.62 () {54 vars in 921 bytes} [Wed Aug 24 06:19:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:19:08.979585 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5ed782-0787-4a67-ae0b-5d67e0300927 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:19:08.985513 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5ed782-0787-4a67-ae0b-5d67e0300927 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:19:08.985513 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5ed782-0787-4a67-ae0b-5d67e0300927 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:19:09.005738 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-2e5ed782-0787-4a67-ae0b-5d67e0300927 None None] MFA Rules not processed for user `fb6d3743340241f98cacb9e0d063298c`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:19:09.007031 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-2e5ed782-0787-4a67-ae0b-5d67e0300927 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:19:09.016697 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1106/2213] 10.222.0.62 () {58 vars in 1018 bytes} [Wed Aug 24 06:19:08 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:19:09.028117 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab43eda-bf1d-4b3e-b010-f3bf778e74f5 None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:19:09.030036 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-bab43eda-bf1d-4b3e-b010-f3bf778e74f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:19:09.044099 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab43eda-bf1d-4b3e-b010-f3bf778e74f5 None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:19:09.047737 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bf55fb-dc46-407d-893d-dad06ef31286 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bb6c79db75e4d169331c02133d1ec32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:19:09.048340 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bf55fb-dc46-407d-893d-dad06ef31286 service neutron] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:19:09.048658 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bf55fb-dc46-407d-893d-dad06ef31286 service neutron] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:19:09.048936 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bf55fb-dc46-407d-893d-dad06ef31286 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:19:09.061496 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bf55fb-dc46-407d-893d-dad06ef31286 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:19:09.062945 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bf55fb-dc46-407d-893d-dad06ef31286 service neutron] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:19:09.075503 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1108/2214] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:19:09 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:19:15.102448 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867e63fb-e268-4278-b433-1eeaaf117669 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:19:15.102936 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867e63fb-e268-4278-b433-1eeaaf117669 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:19:15.103027 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867e63fb-e268-4278-b433-1eeaaf117669 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:19:15.121762 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-867e63fb-e268-4278-b433-1eeaaf117669 None None] MFA Rules not processed for user `b82726a12436435cb931b986ef884c2e`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:19:15.122958 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-867e63fb-e268-4278-b433-1eeaaf117669 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:19:15.130084 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1107/2215] 10.222.0.62 () {58 vars in 1009 bytes} [Wed Aug 24 06:19:15 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:19:15.142378 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bf55fb-dc46-407d-893d-dad06ef31286 service neutron] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:19:15.154074 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bf55fb-dc46-407d-893d-dad06ef31286 service neutron] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:19:15.159605 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e93f2ab-ae44-422a-940b-440f7d64c093 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14d4fed04fb144d08918bd5d4f808d23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:19:15.160202 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e93f2ab-ae44-422a-940b-440f7d64c093 service swift] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:19:15.160471 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e93f2ab-ae44-422a-940b-440f7d64c093 service swift] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:19:15.160943 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e93f2ab-ae44-422a-940b-440f7d64c093 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:19:15.169687 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e93f2ab-ae44-422a-940b-440f7d64c093 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:19:15.171288 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e93f2ab-ae44-422a-940b-440f7d64c093 service swift] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:19:15.180117 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1109/2216] 10.222.0.62 () {60 vars in 1384 bytes} [Wed Aug 24 06:19:15 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 06:19:25.116565 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=43 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:19:25.176793 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=61 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:19:44.147390 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867e63fb-e268-4278-b433-1eeaaf117669 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:19:44.148812 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-867e63fb-e268-4278-b433-1eeaaf117669 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:19:44.219766 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867e63fb-e268-4278-b433-1eeaaf117669 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:19:44.225981 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22db13f-188f-4a09-84fd-ab439a8e8c53 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abe9d8c9291845dfa4ef55034e5c9e32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:19:44.226596 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22db13f-188f-4a09-84fd-ab439a8e8c53 service placement] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:19:44.226871 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22db13f-188f-4a09-84fd-ab439a8e8c53 service placement] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:19:44.227582 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22db13f-188f-4a09-84fd-ab439a8e8c53 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:19:44.229048 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-c22db13f-188f-4a09-84fd-ab439a8e8c53 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:19:44.244360 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22db13f-188f-4a09-84fd-ab439a8e8c53 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:19:44.246621 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22db13f-188f-4a09-84fd-ab439a8e8c53 service placement] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:19:44.258502 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1108/2217] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:19:44 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:19:48.741884 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660336b3-8c55-4205-bd3f-d86da7da1ee4 None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:19:48.743767 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660336b3-8c55-4205-bd3f-d86da7da1ee4 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:19:48.743767 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660336b3-8c55-4205-bd3f-d86da7da1ee4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:19:48.761665 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-660336b3-8c55-4205-bd3f-d86da7da1ee4 None None] MFA Rules not processed for user `b82726a12436435cb931b986ef884c2e`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:19:48.762937 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-660336b3-8c55-4205-bd3f-d86da7da1ee4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:19:48.770057 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1110/2218] 10.222.0.62 () {58 vars in 967 bytes} [Wed Aug 24 06:19:48 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:19:48.782759 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22db13f-188f-4a09-84fd-ab439a8e8c53 service placement] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:19:48.794977 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22db13f-188f-4a09-84fd-ab439a8e8c53 service placement] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:19:48.800254 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397b2a2c-8e4e-48fc-96a8-8ba7ff34c295 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14d4fed04fb144d08918bd5d4f808d23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:19:48.800913 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397b2a2c-8e4e-48fc-96a8-8ba7ff34c295 service swift] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:19:48.801193 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397b2a2c-8e4e-48fc-96a8-8ba7ff34c295 service swift] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:19:48.801714 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397b2a2c-8e4e-48fc-96a8-8ba7ff34c295 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:19:48.810336 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397b2a2c-8e4e-48fc-96a8-8ba7ff34c295 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:19:48.811933 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397b2a2c-8e4e-48fc-96a8-8ba7ff34c295 service swift] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:19:48.820237 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1109/2219] 10.222.0.62 () {60 vars in 1384 bytes} [Wed Aug 24 06:19:48 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 06:19:58.756962 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=3 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:19:58.816953 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=38 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:20:05.793565 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3dfd0a-637e-47c0-ab67-995712787907 None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:20:05.794137 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3dfd0a-637e-47c0-ab67-995712787907 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:20:05.794137 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3dfd0a-637e-47c0-ab67-995712787907 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:20:05.813674 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-2e3dfd0a-637e-47c0-ab67-995712787907 None None] MFA Rules not processed for user `4cb18bdb6d404cd5b303ce0a29212e4f`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:20:05.814903 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-2e3dfd0a-637e-47c0-ab67-995712787907 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:20:05.874198 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1111/2220] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:20:05 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:20:15.867099 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=25 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:20:17.554217 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c8607c-7d36-42b7-8753-1e2d5b5fa54d None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:20:17.554743 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c8607c-7d36-42b7-8753-1e2d5b5fa54d None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:20:17.554743 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c8607c-7d36-42b7-8753-1e2d5b5fa54d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:20:17.574853 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-94c8607c-7d36-42b7-8753-1e2d5b5fa54d None None] MFA Rules not processed for user `069dc6f771a74dd4ab22d9d5e68e3cc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:20:17.576077 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-94c8607c-7d36-42b7-8753-1e2d5b5fa54d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:20:17.634742 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1110/2221] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:20:17 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:20:27.627825 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=25 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:26.880022 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec06038c-3aa0-4f10-9848-9eff49879935 None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:26.880605 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec06038c-3aa0-4f10-9848-9eff49879935 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:26.880605 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec06038c-3aa0-4f10-9848-9eff49879935 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:26.901146 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-ec06038c-3aa0-4f10-9848-9eff49879935 None None] MFA Rules not processed for user `25908fb02f7f46f5b9e483811302a67e`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:21:26.911987 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-ec06038c-3aa0-4f10-9848-9eff49879935 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:21:26.964354 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1112/2222] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:21:26 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:21:27.000903 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c8607c-7d36-42b7-8753-1e2d5b5fa54d None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:27.002302 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-94c8607c-7d36-42b7-8753-1e2d5b5fa54d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:21:27.014485 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c8607c-7d36-42b7-8753-1e2d5b5fa54d None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:27.017991 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351fef54-e0a5-4068-a441-877a3048ae2b service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7d52162cbdc4218b577ecc1e69a88c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:27.018577 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351fef54-e0a5-4068-a441-877a3048ae2b service glance] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:27.018850 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351fef54-e0a5-4068-a441-877a3048ae2b service glance] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:27.019117 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351fef54-e0a5-4068-a441-877a3048ae2b service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:27.020797 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-351fef54-e0a5-4068-a441-877a3048ae2b service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:21:27.033605 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351fef54-e0a5-4068-a441-877a3048ae2b service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:27.035008 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351fef54-e0a5-4068-a441-877a3048ae2b service glance] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:27.045702 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1111/2223] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:21:26 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:21:29.101655 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec06038c-3aa0-4f10-9848-9eff49879935 None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:29.111860 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec06038c-3aa0-4f10-9848-9eff49879935 None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:29.115721 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db11eb81-fae8-4bb2-8a0b-3ff8ee3982b9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bb6c79db75e4d169331c02133d1ec32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:29.116328 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db11eb81-fae8-4bb2-8a0b-3ff8ee3982b9 service neutron] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:29.116641 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db11eb81-fae8-4bb2-8a0b-3ff8ee3982b9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:29.121013 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db11eb81-fae8-4bb2-8a0b-3ff8ee3982b9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:29.121013 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-db11eb81-fae8-4bb2-8a0b-3ff8ee3982b9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:21:29.132872 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db11eb81-fae8-4bb2-8a0b-3ff8ee3982b9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:29.134294 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db11eb81-fae8-4bb2-8a0b-3ff8ee3982b9 service neutron] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:29.145740 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1113/2224] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:21:29 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:21:31.327662 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351fef54-e0a5-4068-a441-877a3048ae2b service glance] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:31.329009 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-351fef54-e0a5-4068-a441-877a3048ae2b service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:21:31.342510 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351fef54-e0a5-4068-a441-877a3048ae2b service glance] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:31.347921 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1cbc47-bd35-4d12-8dc8-688750553e9d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb6d3743340241f98cacb9e0d063298c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:31.348520 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1cbc47-bd35-4d12-8dc8-688750553e9d service nova] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:31.348813 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1cbc47-bd35-4d12-8dc8-688750553e9d service nova] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:31.349085 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1cbc47-bd35-4d12-8dc8-688750553e9d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:31.350720 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-fd1cbc47-bd35-4d12-8dc8-688750553e9d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:21:31.360467 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1cbc47-bd35-4d12-8dc8-688750553e9d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:31.362156 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1cbc47-bd35-4d12-8dc8-688750553e9d service nova] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:31.371620 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1112/2225] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:21:31 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:21:39.141131 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=31 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:39.567764 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db11eb81-fae8-4bb2-8a0b-3ff8ee3982b9 service neutron] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:39.580179 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db11eb81-fae8-4bb2-8a0b-3ff8ee3982b9 service neutron] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:39.585808 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de8496a-0f21-4244-a518-e4c1e8118407 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb6d3743340241f98cacb9e0d063298c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:39.586406 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de8496a-0f21-4244-a518-e4c1e8118407 service nova] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:39.586687 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de8496a-0f21-4244-a518-e4c1e8118407 service nova] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:39.587698 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de8496a-0f21-4244-a518-e4c1e8118407 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:39.588805 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-7de8496a-0f21-4244-a518-e4c1e8118407 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:21:39.670449 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de8496a-0f21-4244-a518-e4c1e8118407 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:39.672295 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de8496a-0f21-4244-a518-e4c1e8118407 service nova] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:39.684904 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1114/2226] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:21:39 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:21:41.367192 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=12 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:42.790204 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1cbc47-bd35-4d12-8dc8-688750553e9d service nova] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:42.791699 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-fd1cbc47-bd35-4d12-8dc8-688750553e9d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:21:42.806509 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1cbc47-bd35-4d12-8dc8-688750553e9d service nova] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:42.812926 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8b0dd5-96bc-4e86-9185-2e70dc742352 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb6d3743340241f98cacb9e0d063298c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:42.813534 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8b0dd5-96bc-4e86-9185-2e70dc742352 service nova] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:42.813805 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8b0dd5-96bc-4e86-9185-2e70dc742352 service nova] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:42.814070 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8b0dd5-96bc-4e86-9185-2e70dc742352 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:42.815786 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-1e8b0dd5-96bc-4e86-9185-2e70dc742352 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:21:42.826629 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8b0dd5-96bc-4e86-9185-2e70dc742352 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:42.828782 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8b0dd5-96bc-4e86-9185-2e70dc742352 service nova] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:42.838012 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1113/2227] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:21:42 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:21:49.677438 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=28 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:52.834103 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=6 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:27.844093 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de8496a-0f21-4244-a518-e4c1e8118407 service nova] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:27.845729 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-7de8496a-0f21-4244-a518-e4c1e8118407 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:22:27.902392 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de8496a-0f21-4244-a518-e4c1e8118407 service nova] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:27.906265 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8517b107-01a7-4fa4-bec0-4014f226a1ba service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c4def55ac0e48b29836bc5c64bc7402', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:27.906876 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8517b107-01a7-4fa4-bec0-4014f226a1ba service cinder] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:27.907161 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8517b107-01a7-4fa4-bec0-4014f226a1ba service cinder] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:27.907450 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8517b107-01a7-4fa4-bec0-4014f226a1ba service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:27.909229 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-8517b107-01a7-4fa4-bec0-4014f226a1ba service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:22:27.970304 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8517b107-01a7-4fa4-bec0-4014f226a1ba service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:27.972016 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8517b107-01a7-4fa4-bec0-4014f226a1ba service cinder] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:27.981930 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1115/2228] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:22:27 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:22:34.693445 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8b0dd5-96bc-4e86-9185-2e70dc742352 service nova] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:34.703526 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8b0dd5-96bc-4e86-9185-2e70dc742352 service nova] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:34.708383 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cd004f-6b95-447a-8a7c-920d7b8f9c51 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bb6c79db75e4d169331c02133d1ec32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:34.709035 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cd004f-6b95-447a-8a7c-920d7b8f9c51 service neutron] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:34.709439 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cd004f-6b95-447a-8a7c-920d7b8f9c51 service neutron] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:34.709604 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cd004f-6b95-447a-8a7c-920d7b8f9c51 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:34.711812 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-42cd004f-6b95-447a-8a7c-920d7b8f9c51 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:22:34.722832 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42cd004f-6b95-447a-8a7c-920d7b8f9c51 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:34.724355 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42cd004f-6b95-447a-8a7c-920d7b8f9c51 service neutron] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:34.733744 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1114/2229] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:22:34 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 06:22:37.977413 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=48 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:44.729386 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=6 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:23:20.423572 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8517b107-01a7-4fa4-bec0-4014f226a1ba service cinder] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:23:20.433173 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8517b107-01a7-4fa4-bec0-4014f226a1ba service cinder] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:23:20.436824 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f66adc-de98-41cc-bf65-a92731095bda service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c4def55ac0e48b29836bc5c64bc7402', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:23:20.437447 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f66adc-de98-41cc-bf65-a92731095bda service cinder] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:23:20.437733 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f66adc-de98-41cc-bf65-a92731095bda service cinder] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:23:20.438005 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f66adc-de98-41cc-bf65-a92731095bda service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:23:20.439705 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-a7f66adc-de98-41cc-bf65-a92731095bda service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:23:20.450573 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f66adc-de98-41cc-bf65-a92731095bda service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:23:20.451992 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f66adc-de98-41cc-bf65-a92731095bda service cinder] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:23:20.461510 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1116/2230] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:23:20 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:23:30.456744 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=6 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:23:35.344157 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cd004f-6b95-447a-8a7c-920d7b8f9c51 service neutron] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:23:35.345693 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-42cd004f-6b95-447a-8a7c-920d7b8f9c51 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:23:35.357340 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cd004f-6b95-447a-8a7c-920d7b8f9c51 service neutron] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:23:35.360933 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8658e28-d5eb-4497-ad32-d18a7a45d272 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c4def55ac0e48b29836bc5c64bc7402', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:23:35.361523 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8658e28-d5eb-4497-ad32-d18a7a45d272 service cinder] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:23:35.361798 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8658e28-d5eb-4497-ad32-d18a7a45d272 service cinder] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:23:35.362067 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8658e28-d5eb-4497-ad32-d18a7a45d272 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:23:35.363705 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-d8658e28-d5eb-4497-ad32-d18a7a45d272 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:23:35.374059 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8658e28-d5eb-4497-ad32-d18a7a45d272 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:23:35.375462 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8658e28-d5eb-4497-ad32-d18a7a45d272 service cinder] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:23:35.384762 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1115/2231] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:23:35 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:23:45.380829 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=6 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:25:34.048589 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f66adc-de98-41cc-bf65-a92731095bda service cinder] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:25:34.050015 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-a7f66adc-de98-41cc-bf65-a92731095bda service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:25:34.135094 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f66adc-de98-41cc-bf65-a92731095bda service cinder] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:25:34.140390 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0946add-5e65-4581-afe9-3fff85b33f8a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abe9d8c9291845dfa4ef55034e5c9e32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:25:34.141059 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0946add-5e65-4581-afe9-3fff85b33f8a service placement] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:25:34.141366 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0946add-5e65-4581-afe9-3fff85b33f8a service placement] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:25:34.141640 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0946add-5e65-4581-afe9-3fff85b33f8a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:25:34.150245 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0946add-5e65-4581-afe9-3fff85b33f8a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:25:34.151816 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0946add-5e65-4581-afe9-3fff85b33f8a service placement] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:25:34.161933 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1117/2232] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:25:34 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:25:44.157836 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=37 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:26:30.719919 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8658e28-d5eb-4497-ad32-d18a7a45d272 service cinder] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:26:30.749650 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8658e28-d5eb-4497-ad32-d18a7a45d272 service cinder] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:26:30.753980 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447dcb9e-e331-497c-8b27-dec8bbcc2b81 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c4def55ac0e48b29836bc5c64bc7402', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:26:30.754585 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447dcb9e-e331-497c-8b27-dec8bbcc2b81 service cinder] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:26:30.754868 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447dcb9e-e331-497c-8b27-dec8bbcc2b81 service cinder] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:26:30.755141 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447dcb9e-e331-497c-8b27-dec8bbcc2b81 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:26:30.764120 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447dcb9e-e331-497c-8b27-dec8bbcc2b81 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:26:30.765593 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447dcb9e-e331-497c-8b27-dec8bbcc2b81 service cinder] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:26:30.775253 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1116/2233] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:26:30 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:26:35.283174 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0946add-5e65-4581-afe9-3fff85b33f8a service placement] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:26:35.293206 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0946add-5e65-4581-afe9-3fff85b33f8a service placement] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:26:35.296838 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559598bc-2f62-48d8-83da-177cc71b85da service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c4def55ac0e48b29836bc5c64bc7402', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:26:35.297439 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559598bc-2f62-48d8-83da-177cc71b85da service cinder] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:26:35.297721 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559598bc-2f62-48d8-83da-177cc71b85da service cinder] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:26:35.297995 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559598bc-2f62-48d8-83da-177cc71b85da service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:26:35.316047 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559598bc-2f62-48d8-83da-177cc71b85da service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:26:35.317534 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559598bc-2f62-48d8-83da-177cc71b85da service cinder] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:26:35.339413 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1118/2234] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:26:35 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:26:38.774188 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447dcb9e-e331-497c-8b27-dec8bbcc2b81 service cinder] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:26:38.789003 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447dcb9e-e331-497c-8b27-dec8bbcc2b81 service cinder] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:26:38.795168 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0950f4-8c68-415e-bf99-be588497368a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb6d3743340241f98cacb9e0d063298c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:26:38.795875 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0950f4-8c68-415e-bf99-be588497368a service nova] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:26:38.796157 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0950f4-8c68-415e-bf99-be588497368a service nova] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:26:38.805577 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0950f4-8c68-415e-bf99-be588497368a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:26:38.805577 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-3a0950f4-8c68-415e-bf99-be588497368a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:26:38.827947 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0950f4-8c68-415e-bf99-be588497368a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:26:38.829699 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0950f4-8c68-415e-bf99-be588497368a service nova] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:26:38.841367 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1117/2235] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:26:38 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:26:38.957818 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559598bc-2f62-48d8-83da-177cc71b85da service cinder] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:26:38.977511 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559598bc-2f62-48d8-83da-177cc71b85da service cinder] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:26:38.981452 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d6e78d-73e4-425c-9033-48dfc9b65b75 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bb6c79db75e4d169331c02133d1ec32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:26:38.982052 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d6e78d-73e4-425c-9033-48dfc9b65b75 service neutron] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:26:38.982333 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d6e78d-73e4-425c-9033-48dfc9b65b75 service neutron] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:26:38.982608 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d6e78d-73e4-425c-9033-48dfc9b65b75 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:26:38.995463 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d6e78d-73e4-425c-9033-48dfc9b65b75 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:26:38.997049 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d6e78d-73e4-425c-9033-48dfc9b65b75 service neutron] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:26:39.008705 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1119/2236] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:26:38 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:26:39.391338 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0950f4-8c68-415e-bf99-be588497368a service nova] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:26:39.404126 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0950f4-8c68-415e-bf99-be588497368a service nova] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:26:39.422691 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ed4ee0-da27-4066-9f69-b4cc0dae7ccb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb6d3743340241f98cacb9e0d063298c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:26:39.422691 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ed4ee0-da27-4066-9f69-b4cc0dae7ccb service nova] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:26:39.422691 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ed4ee0-da27-4066-9f69-b4cc0dae7ccb service nova] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:26:39.422691 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ed4ee0-da27-4066-9f69-b4cc0dae7ccb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:26:39.423179 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ed4ee0-da27-4066-9f69-b4cc0dae7ccb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:26:39.423179 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ed4ee0-da27-4066-9f69-b4cc0dae7ccb service nova] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:26:39.452731 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1118/2237] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:26:39 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:26:40.233887 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce96ec17-c073-48b0-b2b5-18d2ae4ac3b4 None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:26:40.234368 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce96ec17-c073-48b0-b2b5-18d2ae4ac3b4 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:26:40.234492 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce96ec17-c073-48b0-b2b5-18d2ae4ac3b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:26:40.267272 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-ce96ec17-c073-48b0-b2b5-18d2ae4ac3b4 None None] MFA Rules not processed for user `b82726a12436435cb931b986ef884c2e`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:26:40.269143 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-ce96ec17-c073-48b0-b2b5-18d2ae4ac3b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:26:40.276775 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1120/2238] 10.222.0.62 () {58 vars in 967 bytes} [Wed Aug 24 06:26:40 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:26:40.433202 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ed4ee0-da27-4066-9f69-b4cc0dae7ccb service nova] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:26:40.444938 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ed4ee0-da27-4066-9f69-b4cc0dae7ccb service nova] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:26:40.450057 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe37b9ef-b521-4679-b9ab-5656416901bd service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14d4fed04fb144d08918bd5d4f808d23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:26:40.450669 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe37b9ef-b521-4679-b9ab-5656416901bd service swift] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:26:40.450951 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe37b9ef-b521-4679-b9ab-5656416901bd service swift] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:26:40.451227 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe37b9ef-b521-4679-b9ab-5656416901bd service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:26:40.459644 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe37b9ef-b521-4679-b9ab-5656416901bd service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:26:40.461191 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe37b9ef-b521-4679-b9ab-5656416901bd service swift] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:26:40.469206 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1119/2239] 10.222.0.62 () {60 vars in 1384 bytes} [Wed Aug 24 06:26:40 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 06:26:41.608958 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce96ec17-c073-48b0-b2b5-18d2ae4ac3b4 None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:26:41.618242 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce96ec17-c073-48b0-b2b5-18d2ae4ac3b4 None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:26:41.621888 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9875dd7d-2b8d-44aa-a9b8-affffecae5db service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c4def55ac0e48b29836bc5c64bc7402', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:26:41.622497 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9875dd7d-2b8d-44aa-a9b8-affffecae5db service cinder] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:26:41.622770 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9875dd7d-2b8d-44aa-a9b8-affffecae5db service cinder] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:26:41.623040 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9875dd7d-2b8d-44aa-a9b8-affffecae5db service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:26:41.632599 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9875dd7d-2b8d-44aa-a9b8-affffecae5db service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:26:41.636517 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9875dd7d-2b8d-44aa-a9b8-affffecae5db service cinder] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:26:41.643906 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1121/2240] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:26:41 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:26:41.762177 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe37b9ef-b521-4679-b9ab-5656416901bd service swift] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:26:41.772682 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe37b9ef-b521-4679-b9ab-5656416901bd service swift] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:26:41.776294 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445bba34-c63a-4600-9143-c845b4ac188d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bb6c79db75e4d169331c02133d1ec32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:26:41.776969 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445bba34-c63a-4600-9143-c845b4ac188d service neutron] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:26:41.777248 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445bba34-c63a-4600-9143-c845b4ac188d service neutron] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:26:41.777518 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445bba34-c63a-4600-9143-c845b4ac188d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:26:41.779236 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-445bba34-c63a-4600-9143-c845b4ac188d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:26:41.864758 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445bba34-c63a-4600-9143-c845b4ac188d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:26:41.866100 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445bba34-c63a-4600-9143-c845b4ac188d service neutron] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:26:41.885394 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1120/2241] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:26:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:26:45.657185 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9875dd7d-2b8d-44aa-a9b8-affffecae5db service cinder] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:26:45.669493 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9875dd7d-2b8d-44aa-a9b8-affffecae5db service cinder] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:26:45.675076 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ff9030-d78b-4814-9230-31df1deef30a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abe9d8c9291845dfa4ef55034e5c9e32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:26:45.675686 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ff9030-d78b-4814-9230-31df1deef30a service placement] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:26:45.675955 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ff9030-d78b-4814-9230-31df1deef30a service placement] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:26:45.676270 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ff9030-d78b-4814-9230-31df1deef30a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:26:45.678033 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-f2ff9030-d78b-4814-9230-31df1deef30a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:26:45.690473 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ff9030-d78b-4814-9230-31df1deef30a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:26:45.692073 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ff9030-d78b-4814-9230-31df1deef30a service placement] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:26:45.704621 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1122/2242] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:26:45 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:26:49.539195 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445bba34-c63a-4600-9143-c845b4ac188d service neutron] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:26:49.552702 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445bba34-c63a-4600-9143-c845b4ac188d service neutron] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:26:49.558326 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e055cb99-92a2-411d-91ec-a147d698725f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abe9d8c9291845dfa4ef55034e5c9e32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:26:49.559026 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e055cb99-92a2-411d-91ec-a147d698725f service placement] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:26:49.559390 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e055cb99-92a2-411d-91ec-a147d698725f service placement] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:26:49.559724 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e055cb99-92a2-411d-91ec-a147d698725f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:26:49.561587 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-e055cb99-92a2-411d-91ec-a147d698725f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:26:49.574155 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e055cb99-92a2-411d-91ec-a147d698725f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:26:49.575837 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e055cb99-92a2-411d-91ec-a147d698725f service placement] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:26:49.586878 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1121/2243] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:26:49 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:26:49.720787 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ff9030-d78b-4814-9230-31df1deef30a service placement] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:26:49.722420 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-f2ff9030-d78b-4814-9230-31df1deef30a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:26:49.738199 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ff9030-d78b-4814-9230-31df1deef30a service placement] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:26:49.743097 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d87ecf-9daf-4f36-a717-629af24fa8f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:26:49.743727 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d87ecf-9daf-4f36-a717-629af24fa8f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:26:49.744006 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d87ecf-9daf-4f36-a717-629af24fa8f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:26:49.744277 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d87ecf-9daf-4f36-a717-629af24fa8f6 admin admin] PATH_INFO: `/v3/users/d3a03b5f82724f37a2134554d42b2925` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:26:49.745480 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d87ecf-9daf-4f36-a717-629af24fa8f6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d3a03b5f82724f37a2134554d42b2925)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:26:49.747868 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d87ecf-9daf-4f36-a717-629af24fa8f6 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:26:49.990865 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-f1d87ecf-9daf-4f36-a717-629af24fa8f6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd3a03b5f82724f37a2134554d42b2925'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:26:49.998161 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-f1d87ecf-9daf-4f36-a717-629af24fa8f6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd3a03b5f82724f37a2134554d42b2925'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:26:49.998832 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-f1d87ecf-9daf-4f36-a717-629af24fa8f6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd3a03b5f82724f37a2134554d42b2925'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:26:49.999438 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-f1d87ecf-9daf-4f36-a717-629af24fa8f6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd3a03b5f82724f37a2134554d42b2925'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:26:50.079183 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-f1d87ecf-9daf-4f36-a717-629af24fa8f6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd3a03b5f82724f37a2134554d42b2925'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:26:50.098396 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1123/2244] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Aug 24 06:26:49 2022] DELETE /identity/v3/users/d3a03b5f82724f37a2134554d42b2925 => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:26:50.333110 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e055cb99-92a2-411d-91ec-a147d698725f service placement] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:26:50.344854 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e055cb99-92a2-411d-91ec-a147d698725f service placement] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:26:50.354979 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:26:50.406411 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d4ef20-c84c-4cb7-b359-a9a14fd187c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:26:50.407049 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d4ef20-c84c-4cb7-b359-a9a14fd187c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:26:50.407365 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d4ef20-c84c-4cb7-b359-a9a14fd187c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:26:50.407766 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d4ef20-c84c-4cb7-b359-a9a14fd187c4 admin admin] PATH_INFO: `/v3/projects/e2adcef70b1f41d19ddee6811b47be41` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:26:50.409518 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d4ef20-c84c-4cb7-b359-a9a14fd187c4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e2adcef70b1f41d19ddee6811b47be41)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:26:50.411492 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d4ef20-c84c-4cb7-b359-a9a14fd187c4 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:26:50.512198 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-19d4ef20-c84c-4cb7-b359-a9a14fd187c4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e2adcef70b1f41d19ddee6811b47be41', 'request_id': 'req-19d4ef20-c84c-4cb7-b359-a9a14fd187c4'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:26:50.518205 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-19d4ef20-c84c-4cb7-b359-a9a14fd187c4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e2adcef70b1f41d19ddee6811b47be41', 'request_id': 'req-19d4ef20-c84c-4cb7-b359-a9a14fd187c4'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:26:50.524434 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-19d4ef20-c84c-4cb7-b359-a9a14fd187c4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e2adcef70b1f41d19ddee6811b47be41', 'request_id': 'req-19d4ef20-c84c-4cb7-b359-a9a14fd187c4'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:26:50.599200 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-19d4ef20-c84c-4cb7-b359-a9a14fd187c4 admin admin] The token cache is being invalidate because project e2adcef70b1f41d19ddee6811b47be41 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81852) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:26:50.599846 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-19d4ef20-c84c-4cb7-b359-a9a14fd187c4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:26:50.600143 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-19d4ef20-c84c-4cb7-b359-a9a14fd187c4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:26:50.601737 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1122/2245] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Aug 24 06:26:50 2022] DELETE /identity/v3/projects/e2adcef70b1f41d19ddee6811b47be41 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:26:50.605785 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d87ecf-9daf-4f36-a717-629af24fa8f6 admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:26:50.607181 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-f1d87ecf-9daf-4f36-a717-629af24fa8f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:26:50.620028 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d87ecf-9daf-4f36-a717-629af24fa8f6 admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:26:50.656941 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:26:50.675186 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebbe710-42e3-4965-a583-8632741b9c59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:26:50.675818 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebbe710-42e3-4965-a583-8632741b9c59 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:26:50.676097 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebbe710-42e3-4965-a583-8632741b9c59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:26:50.676369 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebbe710-42e3-4965-a583-8632741b9c59 admin admin] PATH_INFO: `/v3/users/d2b44c8575f54d6a82f40b9c3b70c38e` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:26:50.677519 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebbe710-42e3-4965-a583-8632741b9c59 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d2b44c8575f54d6a82f40b9c3b70c38e)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:26:50.679434 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebbe710-42e3-4965-a583-8632741b9c59 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:26:50.936103 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-3ebbe710-42e3-4965-a583-8632741b9c59 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd2b44c8575f54d6a82f40b9c3b70c38e'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:26:50.944152 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-3ebbe710-42e3-4965-a583-8632741b9c59 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd2b44c8575f54d6a82f40b9c3b70c38e'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:26:50.944942 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-3ebbe710-42e3-4965-a583-8632741b9c59 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd2b44c8575f54d6a82f40b9c3b70c38e'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:26:50.945586 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-3ebbe710-42e3-4965-a583-8632741b9c59 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd2b44c8575f54d6a82f40b9c3b70c38e'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:26:51.026140 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-3ebbe710-42e3-4965-a583-8632741b9c59 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd2b44c8575f54d6a82f40b9c3b70c38e'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:26:51.045928 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1124/2246] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Aug 24 06:26:50 2022] DELETE /identity/v3/users/d2b44c8575f54d6a82f40b9c3b70c38e => generated 0 bytes in 441 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 06:26:51.337708 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d4ef20-c84c-4cb7-b359-a9a14fd187c4 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:26:51.339172 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-19d4ef20-c84c-4cb7-b359-a9a14fd187c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:26:51.354517 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d4ef20-c84c-4cb7-b359-a9a14fd187c4 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:26:51.413868 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bf23ea-7901-4723-9b52-bc7fedf350e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:26:51.414502 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bf23ea-7901-4723-9b52-bc7fedf350e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:26:51.414824 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bf23ea-7901-4723-9b52-bc7fedf350e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:26:51.415100 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bf23ea-7901-4723-9b52-bc7fedf350e4 admin admin] PATH_INFO: `/v3/projects/7d37379ce6c24acf83bd68e06fca20f8` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:26:51.416299 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bf23ea-7901-4723-9b52-bc7fedf350e4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7d37379ce6c24acf83bd68e06fca20f8)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:26:51.418246 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bf23ea-7901-4723-9b52-bc7fedf350e4 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:26:51.528286 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:26:51.531307 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-d0bf23ea-7901-4723-9b52-bc7fedf350e4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7d37379ce6c24acf83bd68e06fca20f8', 'request_id': 'req-d0bf23ea-7901-4723-9b52-bc7fedf350e4'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:26:51.537495 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-d0bf23ea-7901-4723-9b52-bc7fedf350e4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7d37379ce6c24acf83bd68e06fca20f8', 'request_id': 'req-d0bf23ea-7901-4723-9b52-bc7fedf350e4'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:26:51.544027 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-d0bf23ea-7901-4723-9b52-bc7fedf350e4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7d37379ce6c24acf83bd68e06fca20f8', 'request_id': 'req-d0bf23ea-7901-4723-9b52-bc7fedf350e4'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:26:51.654512 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-d0bf23ea-7901-4723-9b52-bc7fedf350e4 admin admin] The token cache is being invalidate because project 7d37379ce6c24acf83bd68e06fca20f8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81852) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:26:51.655169 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-d0bf23ea-7901-4723-9b52-bc7fedf350e4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:26:51.655727 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-d0bf23ea-7901-4723-9b52-bc7fedf350e4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:26:51.657304 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1123/2247] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Aug 24 06:26:51 2022] DELETE /identity/v3/projects/7d37379ce6c24acf83bd68e06fca20f8 => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:26:51.663986 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8277ca35-a466-4e33-8c34-9f5b919cf28c None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:26:51.664383 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8277ca35-a466-4e33-8c34-9f5b919cf28c None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:26:51.664698 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8277ca35-a466-4e33-8c34-9f5b919cf28c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:26:51.728593 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-8277ca35-a466-4e33-8c34-9f5b919cf28c None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:26:51.729836 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-8277ca35-a466-4e33-8c34-9f5b919cf28c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:26:51.792311 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1125/2248] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:26:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:26:51.800890 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac44498-6f3d-46d7-94e8-6a73fc8cc31b None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:26:51.801167 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac44498-6f3d-46d7-94e8-6a73fc8cc31b None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:26:51.801445 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac44498-6f3d-46d7-94e8-6a73fc8cc31b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:26:51.820974 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-7ac44498-6f3d-46d7-94e8-6a73fc8cc31b None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:26:51.822173 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-7ac44498-6f3d-46d7-94e8-6a73fc8cc31b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:26:51.830034 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1124/2249] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:26:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:26:51.834296 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8277ca35-a466-4e33-8c34-9f5b919cf28c None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:26:51.844708 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8277ca35-a466-4e33-8c34-9f5b919cf28c None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:26:51.849747 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c181b4c-3b8a-4cb1-ac86-f936972743e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:26:51.850342 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c181b4c-3b8a-4cb1-ac86-f936972743e0 admin admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:26:51.850619 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c181b4c-3b8a-4cb1-ac86-f936972743e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:26:51.850886 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c181b4c-3b8a-4cb1-ac86-f936972743e0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:26:51.851528 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c181b4c-3b8a-4cb1-ac86-f936972743e0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:26:51.851826 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c181b4c-3b8a-4cb1-ac86-f936972743e0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:26:51.853434 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c181b4c-3b8a-4cb1-ac86-f936972743e0 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:26:51.867346 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1126/2250] 10.222.0.62 () {58 vars in 1153 bytes} [Wed Aug 24 06:26:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:26:51.870426 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac44498-6f3d-46d7-94e8-6a73fc8cc31b None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:26:51.880202 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac44498-6f3d-46d7-94e8-6a73fc8cc31b None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:26:51.885003 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4f7c56-1e86-4dec-9939-df1699fb1331 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:26:51.885625 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4f7c56-1e86-4dec-9939-df1699fb1331 admin admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:26:51.885899 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4f7c56-1e86-4dec-9939-df1699fb1331 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:26:51.886176 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4f7c56-1e86-4dec-9939-df1699fb1331 admin admin] PATH_INFO: `/v3/projects` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:26:51.886808 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4f7c56-1e86-4dec-9939-df1699fb1331 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:26:51.888752 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4f7c56-1e86-4dec-9939-df1699fb1331 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:26:52.014047 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1125/2251] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Aug 24 06:26:51 2022] POST /identity/v3/projects => generated 373 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:26:52.018481 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c181b4c-3b8a-4cb1-ac86-f936972743e0 admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:26:52.028935 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c181b4c-3b8a-4cb1-ac86-f936972743e0 admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:26:52.042675 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:26:52.104774 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c2974b-a5cf-4892-9eb0-e768dbcb7eae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:26:52.105429 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c2974b-a5cf-4892-9eb0-e768dbcb7eae admin admin] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:26:52.110206 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c2974b-a5cf-4892-9eb0-e768dbcb7eae admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:26:52.110206 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c2974b-a5cf-4892-9eb0-e768dbcb7eae admin admin] PATH_INFO: `/v3/users` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:26:52.110206 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c2974b-a5cf-4892-9eb0-e768dbcb7eae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:26:52.110206 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c2974b-a5cf-4892-9eb0-e768dbcb7eae admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:26:52.234311 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1127/2252] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Aug 24 06:26:52 2022] POST /identity/v3/users => generated 287 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:26:52.238806 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4f7c56-1e86-4dec-9939-df1699fb1331 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:26:52.249777 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4f7c56-1e86-4dec-9939-df1699fb1331 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:26:52.254884 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7d05dc-7174-4b59-9759-0074186dd538 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:26:52.259134 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7d05dc-7174-4b59-9759-0074186dd538 admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:26:52.259134 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7d05dc-7174-4b59-9759-0074186dd538 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:26:52.259134 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7d05dc-7174-4b59-9759-0074186dd538 admin admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:26:52.259134 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7d05dc-7174-4b59-9759-0074186dd538 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:26:52.259134 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7d05dc-7174-4b59-9759-0074186dd538 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:26:52.269824 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1126/2253] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Aug 24 06:26:52 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 06:26:52.274024 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c2974b-a5cf-4892-9eb0-e768dbcb7eae admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:26:52.285125 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c2974b-a5cf-4892-9eb0-e768dbcb7eae admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:26:52.289988 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493156b1-d1e0-4ee0-89fa-fa991ce59675 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:26:52.290599 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493156b1-d1e0-4ee0-89fa-fa991ce59675 admin admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:26:52.290878 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493156b1-d1e0-4ee0-89fa-fa991ce59675 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:26:52.291156 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493156b1-d1e0-4ee0-89fa-fa991ce59675 admin admin] PATH_INFO: `/v3/roles` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:26:52.291864 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493156b1-d1e0-4ee0-89fa-fa991ce59675 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:26:52.293646 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493156b1-d1e0-4ee0-89fa-fa991ce59675 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:26:52.304273 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1128/2254] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Aug 24 06:26:52 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 06:26:52.308822 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7d05dc-7174-4b59-9759-0074186dd538 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:26:52.319203 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7d05dc-7174-4b59-9759-0074186dd538 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:26:52.324037 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba81852-1ad0-4a30-bc26-92862a33da6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:26:52.324745 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba81852-1ad0-4a30-bc26-92862a33da6e admin admin] REQUEST_METHOD: `PUT` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:26:52.325154 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba81852-1ad0-4a30-bc26-92862a33da6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:26:52.325436 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba81852-1ad0-4a30-bc26-92862a33da6e admin admin] PATH_INFO: `/v3/projects/df659b3447644e7dbe0f124b881e4344/users/27d9cb186f8f40498141aa20db23e9d0/roles/580aa7dbb38d4cff8f9d1bb8983f99ef` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:26:52.343648 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba81852-1ad0-4a30-bc26-92862a33da6e admin admin] RBAC: Authorizing `identity:create_grant(project_id=df659b3447644e7dbe0f124b881e4344, user_id=27d9cb186f8f40498141aa20db23e9d0, role_id=580aa7dbb38d4cff8f9d1bb8983f99ef)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:26:52.346525 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba81852-1ad0-4a30-bc26-92862a33da6e admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:26:52.450956 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1127/2255] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Aug 24 06:26:52 2022] PUT /identity/v3/projects/df659b3447644e7dbe0f124b881e4344/users/27d9cb186f8f40498141aa20db23e9d0/roles/580aa7dbb38d4cff8f9d1bb8983f99ef => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:26:52.457099 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fa511c-ad01-44a9-aac1-e590a2232bc5 None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:26:52.457386 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fa511c-ad01-44a9-aac1-e590a2232bc5 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:26:52.457658 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fa511c-ad01-44a9-aac1-e590a2232bc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:26:52.477176 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-49fa511c-ad01-44a9-aac1-e590a2232bc5 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:26:52.478474 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-49fa511c-ad01-44a9-aac1-e590a2232bc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:26:52.537163 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1129/2256] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:26:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:26:52.548730 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba81852-1ad0-4a30-bc26-92862a33da6e admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:26:52.550436 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-cba81852-1ad0-4a30-bc26-92862a33da6e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:26:52.617997 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba81852-1ad0-4a30-bc26-92862a33da6e admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:26:52.621674 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c89c124-7c1c-4b0e-beb9-2087097a5026 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bb6c79db75e4d169331c02133d1ec32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:26:52.629433 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c89c124-7c1c-4b0e-beb9-2087097a5026 service neutron] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:26:52.629433 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c89c124-7c1c-4b0e-beb9-2087097a5026 service neutron] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:26:52.629433 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c89c124-7c1c-4b0e-beb9-2087097a5026 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:26:52.632555 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c89c124-7c1c-4b0e-beb9-2087097a5026 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:26:52.634015 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c89c124-7c1c-4b0e-beb9-2087097a5026 service neutron] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:26:52.644242 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1128/2257] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:26:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:26:58.269627 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c2c039-9bfb-4119-bb50-914d0c8822ba None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:26:58.270043 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c2c039-9bfb-4119-bb50-914d0c8822ba None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:26:58.270304 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c2c039-9bfb-4119-bb50-914d0c8822ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:26:58.312123 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-d3c2c039-9bfb-4119-bb50-914d0c8822ba None None] MFA Rules not processed for user `27d9cb186f8f40498141aa20db23e9d0`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:26:58.313416 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-d3c2c039-9bfb-4119-bb50-914d0c8822ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:26:58.387002 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1130/2258] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:26:58 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:26:58.391537 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c89c124-7c1c-4b0e-beb9-2087097a5026 service neutron] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:26:58.403376 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c89c124-7c1c-4b0e-beb9-2087097a5026 service neutron] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:26:58.410390 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e341eb-a7da-4ff6-af03-67cfd8074b0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:26:58.410390 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e341eb-a7da-4ff6-af03-67cfd8074b0f admin admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:26:58.410390 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e341eb-a7da-4ff6-af03-67cfd8074b0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:26:58.410390 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e341eb-a7da-4ff6-af03-67cfd8074b0f admin admin] PATH_INFO: `/v3/projects` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:26:58.411014 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e341eb-a7da-4ff6-af03-67cfd8074b0f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:26:58.412392 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e341eb-a7da-4ff6-af03-67cfd8074b0f admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:26:58.513279 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1129/2259] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Aug 24 06:26:58 2022] POST /identity/v3/projects => generated 371 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:26:58.517410 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c2c039-9bfb-4119-bb50-914d0c8822ba None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:26:58.528444 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c2c039-9bfb-4119-bb50-914d0c8822ba None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:26:58.551802 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:26:58.592221 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439c5240-1e9e-4626-a158-5c4763799a39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:26:58.592898 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439c5240-1e9e-4626-a158-5c4763799a39 admin admin] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:26:58.593181 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439c5240-1e9e-4626-a158-5c4763799a39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:26:58.593459 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439c5240-1e9e-4626-a158-5c4763799a39 admin admin] PATH_INFO: `/v3/users` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:26:58.594110 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439c5240-1e9e-4626-a158-5c4763799a39 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:26:58.596080 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439c5240-1e9e-4626-a158-5c4763799a39 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:26:58.686201 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1131/2260] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Aug 24 06:26:58 2022] POST /identity/v3/users => generated 292 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 06:26:58.691004 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e341eb-a7da-4ff6-af03-67cfd8074b0f admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:26:58.702398 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e341eb-a7da-4ff6-af03-67cfd8074b0f admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:26:58.708026 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15dec2a-8668-4afa-950c-ceaee2189337 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:26:58.709036 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15dec2a-8668-4afa-950c-ceaee2189337 admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:26:58.709671 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15dec2a-8668-4afa-950c-ceaee2189337 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:26:58.710187 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15dec2a-8668-4afa-950c-ceaee2189337 admin admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:26:58.711314 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15dec2a-8668-4afa-950c-ceaee2189337 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:26:58.713184 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15dec2a-8668-4afa-950c-ceaee2189337 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:26:58.729730 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1130/2261] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Aug 24 06:26:58 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 06:26:58.734447 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439c5240-1e9e-4626-a158-5c4763799a39 admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:26:58.742795 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439c5240-1e9e-4626-a158-5c4763799a39 admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:26:58.747935 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77647b6a-9774-4930-af87-5a050d270c1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:26:58.748668 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77647b6a-9774-4930-af87-5a050d270c1b admin admin] REQUEST_METHOD: `PUT` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:26:58.748944 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77647b6a-9774-4930-af87-5a050d270c1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:26:58.749221 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77647b6a-9774-4930-af87-5a050d270c1b admin admin] PATH_INFO: `/v3/projects/d77ef3633caf42679fee9233665da60c/users/7fb72e8309fa4d8197e1f592289b05be/roles/49e47cd157d743f6905f1db07ec878a7` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:26:58.768633 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77647b6a-9774-4930-af87-5a050d270c1b admin admin] RBAC: Authorizing `identity:create_grant(project_id=d77ef3633caf42679fee9233665da60c, user_id=7fb72e8309fa4d8197e1f592289b05be, role_id=49e47cd157d743f6905f1db07ec878a7)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:26:58.771622 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77647b6a-9774-4930-af87-5a050d270c1b admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:26:58.889551 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1132/2262] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Aug 24 06:26:58 2022] PUT /identity/v3/projects/d77ef3633caf42679fee9233665da60c/users/7fb72e8309fa4d8197e1f592289b05be/roles/49e47cd157d743f6905f1db07ec878a7 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:26:58.893747 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15dec2a-8668-4afa-950c-ceaee2189337 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:26:58.903887 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15dec2a-8668-4afa-950c-ceaee2189337 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:26:58.922188 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:26:58.967091 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db27877-b681-4c0d-9582-8961ab8e9003 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:26:58.967771 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db27877-b681-4c0d-9582-8961ab8e9003 admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:26:58.968117 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db27877-b681-4c0d-9582-8961ab8e9003 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:26:58.968396 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db27877-b681-4c0d-9582-8961ab8e9003 admin admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:26:58.969095 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db27877-b681-4c0d-9582-8961ab8e9003 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:26:58.971190 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db27877-b681-4c0d-9582-8961ab8e9003 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:26:58.983599 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1131/2263] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Aug 24 06:26:58 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:26:58.988682 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77647b6a-9774-4930-af87-5a050d270c1b admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:26:58.999421 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77647b6a-9774-4930-af87-5a050d270c1b admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:26:59.006960 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d501b2-7466-49c4-9446-2c80a6489e7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:26:59.007646 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d501b2-7466-49c4-9446-2c80a6489e7a admin admin] REQUEST_METHOD: `PUT` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:26:59.007924 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d501b2-7466-49c4-9446-2c80a6489e7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:26:59.008250 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d501b2-7466-49c4-9446-2c80a6489e7a admin admin] PATH_INFO: `/v3/projects/d77ef3633caf42679fee9233665da60c/users/7fb72e8309fa4d8197e1f592289b05be/roles/49e47cd157d743f6905f1db07ec878a7` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:26:59.010235 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d501b2-7466-49c4-9446-2c80a6489e7a admin admin] RBAC: Authorizing `identity:create_grant(project_id=d77ef3633caf42679fee9233665da60c, user_id=7fb72e8309fa4d8197e1f592289b05be, role_id=49e47cd157d743f6905f1db07ec878a7)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:26:59.015520 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d501b2-7466-49c4-9446-2c80a6489e7a admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:26:59.024660 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1133/2264] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Aug 24 06:26:58 2022] PUT /identity/v3/projects/d77ef3633caf42679fee9233665da60c/users/7fb72e8309fa4d8197e1f592289b05be/roles/49e47cd157d743f6905f1db07ec878a7 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:27:04.752363 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5228c1b2-8304-401d-aebb-26cd53f1e7cb None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:27:04.752918 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5228c1b2-8304-401d-aebb-26cd53f1e7cb None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:27:04.753038 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5228c1b2-8304-401d-aebb-26cd53f1e7cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:27:04.791162 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-5228c1b2-8304-401d-aebb-26cd53f1e7cb None None] MFA Rules not processed for user `7fb72e8309fa4d8197e1f592289b05be`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:27:04.792999 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-5228c1b2-8304-401d-aebb-26cd53f1e7cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:27:04.889616 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1132/2265] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:27:04 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:27:04.904687 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d501b2-7466-49c4-9446-2c80a6489e7a admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:27:04.907157 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-64d501b2-7466-49c4-9446-2c80a6489e7a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:27:05.008861 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d501b2-7466-49c4-9446-2c80a6489e7a admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:27:05.016335 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca9a9da-575d-446b-95ab-31f8e707ed23 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb6d3743340241f98cacb9e0d063298c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:27:05.017085 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca9a9da-575d-446b-95ab-31f8e707ed23 service nova] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:27:05.017364 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca9a9da-575d-446b-95ab-31f8e707ed23 service nova] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:27:05.017637 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca9a9da-575d-446b-95ab-31f8e707ed23 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:27:05.085902 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca9a9da-575d-446b-95ab-31f8e707ed23 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:27:05.087528 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca9a9da-575d-446b-95ab-31f8e707ed23 service nova] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:27:05.098280 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1134/2266] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:27:04 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:27:07.983420 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f52d3d-7f48-4e70-a9bc-561ea67774d5 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:27:07.983959 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f52d3d-7f48-4e70-a9bc-561ea67774d5 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:27:07.984221 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f52d3d-7f48-4e70-a9bc-561ea67774d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:27:08.004422 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-68f52d3d-7f48-4e70-a9bc-561ea67774d5 None None] MFA Rules not processed for user `27d9cb186f8f40498141aa20db23e9d0`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:27:08.005870 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-68f52d3d-7f48-4e70-a9bc-561ea67774d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:27:08.012932 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1133/2267] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:27:07 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:27:15.093641 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=89,keystone:INSERT=5 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:27:17.998352 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=57 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:27:37.684007 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca9a9da-575d-446b-95ab-31f8e707ed23 service nova] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:27:37.685806 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-3ca9a9da-575d-446b-95ab-31f8e707ed23 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:27:37.747199 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca9a9da-575d-446b-95ab-31f8e707ed23 service nova] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:27:37.750773 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd71b57-b68f-4fb1-a909-5be227312487 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bb6c79db75e4d169331c02133d1ec32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:27:37.751371 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd71b57-b68f-4fb1-a909-5be227312487 service neutron] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:27:37.751648 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd71b57-b68f-4fb1-a909-5be227312487 service neutron] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:27:37.751918 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd71b57-b68f-4fb1-a909-5be227312487 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:27:37.753659 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-2bd71b57-b68f-4fb1-a909-5be227312487 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:27:37.762866 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd71b57-b68f-4fb1-a909-5be227312487 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:27:37.764275 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd71b57-b68f-4fb1-a909-5be227312487 service neutron] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:27:37.772204 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1135/2268] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:27:37 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:27:47.768788 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=26 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:28:20.227039 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f52d3d-7f48-4e70-a9bc-561ea67774d5 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:28:20.240657 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f52d3d-7f48-4e70-a9bc-561ea67774d5 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:28:20.246597 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bb390b-ab31-4d43-9d95-1e21f1f76c18 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb6d3743340241f98cacb9e0d063298c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:28:20.247210 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bb390b-ab31-4d43-9d95-1e21f1f76c18 service nova] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:28:20.247483 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bb390b-ab31-4d43-9d95-1e21f1f76c18 service nova] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:28:20.247749 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bb390b-ab31-4d43-9d95-1e21f1f76c18 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:28:20.249546 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-f0bb390b-ab31-4d43-9d95-1e21f1f76c18 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:28:20.337567 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0bb390b-ab31-4d43-9d95-1e21f1f76c18 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:28:20.339176 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0bb390b-ab31-4d43-9d95-1e21f1f76c18 service nova] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:28:20.358949 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1134/2269] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:28:20 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:28:30.355639 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=41 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:28:35.736860 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd71b57-b68f-4fb1-a909-5be227312487 service neutron] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:28:35.738287 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-2bd71b57-b68f-4fb1-a909-5be227312487 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:28:35.808438 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd71b57-b68f-4fb1-a909-5be227312487 service neutron] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:28:35.812150 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39a0dc4-2782-4cf2-b914-7e63dcb3846f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c4def55ac0e48b29836bc5c64bc7402', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:28:35.812773 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39a0dc4-2782-4cf2-b914-7e63dcb3846f service cinder] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:28:35.813052 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39a0dc4-2782-4cf2-b914-7e63dcb3846f service cinder] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:28:35.813321 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39a0dc4-2782-4cf2-b914-7e63dcb3846f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:28:35.815037 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-a39a0dc4-2782-4cf2-b914-7e63dcb3846f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:28:35.904774 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39a0dc4-2782-4cf2-b914-7e63dcb3846f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:28:35.906243 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39a0dc4-2782-4cf2-b914-7e63dcb3846f service cinder] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:28:35.923941 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1136/2270] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:28:35 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:28:45.921380 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=66 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:39.716250 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bb390b-ab31-4d43-9d95-1e21f1f76c18 service nova] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:39.717711 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-f0bb390b-ab31-4d43-9d95-1e21f1f76c18 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:30:39.786302 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bb390b-ab31-4d43-9d95-1e21f1f76c18 service nova] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:39.793502 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd2f4b0-2506-4859-b781-f208ddc78619 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abe9d8c9291845dfa4ef55034e5c9e32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:39.794112 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd2f4b0-2506-4859-b781-f208ddc78619 service placement] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:39.794390 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd2f4b0-2506-4859-b781-f208ddc78619 service placement] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:39.794663 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd2f4b0-2506-4859-b781-f208ddc78619 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:39.796388 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-5fd2f4b0-2506-4859-b781-f208ddc78619 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:30:39.808921 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd2f4b0-2506-4859-b781-f208ddc78619 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:39.810463 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd2f4b0-2506-4859-b781-f208ddc78619 service placement] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:39.820270 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1135/2271] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:30:39 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:30:49.816165 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=32 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:31:30.586232 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39a0dc4-2782-4cf2-b914-7e63dcb3846f service cinder] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:30.587648 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-a39a0dc4-2782-4cf2-b914-7e63dcb3846f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:31:30.598529 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39a0dc4-2782-4cf2-b914-7e63dcb3846f service cinder] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:30.603196 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b137643-8ff4-4f5a-a116-131acad5b23b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c4def55ac0e48b29836bc5c64bc7402', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:30.603814 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b137643-8ff4-4f5a-a116-131acad5b23b service cinder] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:30.604088 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b137643-8ff4-4f5a-a116-131acad5b23b service cinder] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:30.604357 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b137643-8ff4-4f5a-a116-131acad5b23b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:30.606164 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-0b137643-8ff4-4f5a-a116-131acad5b23b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:31:30.686826 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b137643-8ff4-4f5a-a116-131acad5b23b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:30.688273 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b137643-8ff4-4f5a-a116-131acad5b23b service cinder] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:30.706658 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1137/2272] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:31:30 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:31:34.601168 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd2f4b0-2506-4859-b781-f208ddc78619 service placement] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:34.610654 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd2f4b0-2506-4859-b781-f208ddc78619 service placement] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:34.614278 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9a6eed-50a8-4c96-b2d6-6179f22c336f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bb6c79db75e4d169331c02133d1ec32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:34.614878 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9a6eed-50a8-4c96-b2d6-6179f22c336f service neutron] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:34.615156 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9a6eed-50a8-4c96-b2d6-6179f22c336f service neutron] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:34.616113 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9a6eed-50a8-4c96-b2d6-6179f22c336f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:34.617706 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-2b9a6eed-50a8-4c96-b2d6-6179f22c336f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:31:34.633913 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9a6eed-50a8-4c96-b2d6-6179f22c336f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:34.635341 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9a6eed-50a8-4c96-b2d6-6179f22c336f service neutron] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:34.646318 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1136/2273] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:31:34 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:31:39.762239 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b137643-8ff4-4f5a-a116-131acad5b23b service cinder] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:39.770613 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b137643-8ff4-4f5a-a116-131acad5b23b service cinder] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:39.774522 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e842fc1-a660-49e3-8b40-625af82f467d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c4def55ac0e48b29836bc5c64bc7402', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:39.775110 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e842fc1-a660-49e3-8b40-625af82f467d service cinder] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:39.775386 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e842fc1-a660-49e3-8b40-625af82f467d service cinder] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:39.775653 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e842fc1-a660-49e3-8b40-625af82f467d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:39.777383 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-3e842fc1-a660-49e3-8b40-625af82f467d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:31:39.850335 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e842fc1-a660-49e3-8b40-625af82f467d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:39.851772 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e842fc1-a660-49e3-8b40-625af82f467d service cinder] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:39.860981 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1138/2274] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:31:39 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:31:40.506969 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9a6eed-50a8-4c96-b2d6-6179f22c336f service neutron] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:40.515542 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9a6eed-50a8-4c96-b2d6-6179f22c336f service neutron] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:40.519250 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0ff88d-b6a4-45b2-9296-757aa8f1be11 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bb6c79db75e4d169331c02133d1ec32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:40.519839 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0ff88d-b6a4-45b2-9296-757aa8f1be11 service neutron] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:40.520117 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0ff88d-b6a4-45b2-9296-757aa8f1be11 service neutron] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:40.520388 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0ff88d-b6a4-45b2-9296-757aa8f1be11 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:40.522177 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-6e0ff88d-b6a4-45b2-9296-757aa8f1be11 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:31:40.536061 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0ff88d-b6a4-45b2-9296-757aa8f1be11 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:40.537514 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0ff88d-b6a4-45b2-9296-757aa8f1be11 service neutron] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:40.548809 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1137/2275] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:31:40 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:31:41.721634 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e842fc1-a660-49e3-8b40-625af82f467d service cinder] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:41.730451 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e842fc1-a660-49e3-8b40-625af82f467d service cinder] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:41.734543 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0846191-b8fb-4a89-ac56-1f2907a56a15 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c4def55ac0e48b29836bc5c64bc7402', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:41.735318 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0846191-b8fb-4a89-ac56-1f2907a56a15 service cinder] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:41.735605 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0846191-b8fb-4a89-ac56-1f2907a56a15 service cinder] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:41.735883 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0846191-b8fb-4a89-ac56-1f2907a56a15 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:41.737851 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-d0846191-b8fb-4a89-ac56-1f2907a56a15 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:31:41.797990 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:31:41.823014 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0846191-b8fb-4a89-ac56-1f2907a56a15 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:41.824474 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0846191-b8fb-4a89-ac56-1f2907a56a15 service cinder] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:41.843923 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1139/2276] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:31:41 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:31:50.544083 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=12 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:31:51.859473 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=15 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:32:05.644223 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0ff88d-b6a4-45b2-9296-757aa8f1be11 service neutron] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:32:05.653295 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0ff88d-b6a4-45b2-9296-757aa8f1be11 service neutron] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:32:05.656848 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64077cfa-f5a2-4bd7-a246-c3de9dd722da service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c4def55ac0e48b29836bc5c64bc7402', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:32:05.657438 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64077cfa-f5a2-4bd7-a246-c3de9dd722da service cinder] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:32:05.657711 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64077cfa-f5a2-4bd7-a246-c3de9dd722da service cinder] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:32:05.657976 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64077cfa-f5a2-4bd7-a246-c3de9dd722da service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:32:05.666704 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64077cfa-f5a2-4bd7-a246-c3de9dd722da service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:32:05.668125 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64077cfa-f5a2-4bd7-a246-c3de9dd722da service cinder] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:32:05.676938 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1138/2277] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:32:05 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 33 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:32:15.673014 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=6 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:32:37.116750 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0846191-b8fb-4a89-ac56-1f2907a56a15 service cinder] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:32:37.135002 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0846191-b8fb-4a89-ac56-1f2907a56a15 service cinder] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:32:37.152444 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5208cfdb-25fc-46e1-950f-1d09a10e05a4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb6d3743340241f98cacb9e0d063298c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:32:37.153129 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5208cfdb-25fc-46e1-950f-1d09a10e05a4 service nova] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:32:37.153564 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5208cfdb-25fc-46e1-950f-1d09a10e05a4 service nova] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:32:37.153846 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5208cfdb-25fc-46e1-950f-1d09a10e05a4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:32:37.155685 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-5208cfdb-25fc-46e1-950f-1d09a10e05a4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:32:37.170863 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5208cfdb-25fc-46e1-950f-1d09a10e05a4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:32:37.172769 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5208cfdb-25fc-46e1-950f-1d09a10e05a4 service nova] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:32:37.185797 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1140/2278] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:32:37 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 70 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:32:38.672190 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64077cfa-f5a2-4bd7-a246-c3de9dd722da service cinder] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:32:38.682534 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64077cfa-f5a2-4bd7-a246-c3de9dd722da service cinder] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:32:38.686499 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1390ae2f-2498-4516-9336-ddd02158952f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bb6c79db75e4d169331c02133d1ec32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:32:38.687113 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1390ae2f-2498-4516-9336-ddd02158952f service neutron] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:32:38.687391 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1390ae2f-2498-4516-9336-ddd02158952f service neutron] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:32:38.687664 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1390ae2f-2498-4516-9336-ddd02158952f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:32:38.695383 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1390ae2f-2498-4516-9336-ddd02158952f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:32:38.696957 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1390ae2f-2498-4516-9336-ddd02158952f service neutron] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:32:38.705338 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1139/2279] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:32:38 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:32:47.180100 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=6 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:32:48.701424 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=6 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:33:34.220884 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5208cfdb-25fc-46e1-950f-1d09a10e05a4 service nova] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:33:34.230474 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5208cfdb-25fc-46e1-950f-1d09a10e05a4 service nova] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:33:34.234532 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43b9f21-0521-42d4-ae2b-32c690812b71 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c4def55ac0e48b29836bc5c64bc7402', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:33:34.235167 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43b9f21-0521-42d4-ae2b-32c690812b71 service cinder] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:33:34.235446 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43b9f21-0521-42d4-ae2b-32c690812b71 service cinder] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:33:34.235720 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43b9f21-0521-42d4-ae2b-32c690812b71 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:33:34.245660 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43b9f21-0521-42d4-ae2b-32c690812b71 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:33:34.247169 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43b9f21-0521-42d4-ae2b-32c690812b71 service cinder] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:33:34.257055 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1141/2280] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:33:34 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:33:36.943895 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1390ae2f-2498-4516-9336-ddd02158952f service neutron] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:33:36.945480 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-1390ae2f-2498-4516-9336-ddd02158952f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:33:36.965315 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1390ae2f-2498-4516-9336-ddd02158952f service neutron] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:33:36.974692 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937a8ad2-bedb-4d2f-803f-2346616014d9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb6d3743340241f98cacb9e0d063298c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:33:36.975738 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937a8ad2-bedb-4d2f-803f-2346616014d9 service nova] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:33:36.976250 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937a8ad2-bedb-4d2f-803f-2346616014d9 service nova] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:33:36.976847 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937a8ad2-bedb-4d2f-803f-2346616014d9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:33:36.991949 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937a8ad2-bedb-4d2f-803f-2346616014d9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:33:36.995071 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937a8ad2-bedb-4d2f-803f-2346616014d9 service nova] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:33:37.007635 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1140/2281] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:33:36 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:33:44.252202 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=6 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:33:47.002818 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=6 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:34:37.966465 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43b9f21-0521-42d4-ae2b-32c690812b71 service cinder] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:34:37.976212 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43b9f21-0521-42d4-ae2b-32c690812b71 service cinder] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:34:37.979799 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2ee23b-85ca-444f-9e1e-2b6f1f46761e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bb6c79db75e4d169331c02133d1ec32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:34:37.980396 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2ee23b-85ca-444f-9e1e-2b6f1f46761e service neutron] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:34:37.980712 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2ee23b-85ca-444f-9e1e-2b6f1f46761e service neutron] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:34:37.980988 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2ee23b-85ca-444f-9e1e-2b6f1f46761e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:34:38.040704 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2ee23b-85ca-444f-9e1e-2b6f1f46761e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:34:38.042175 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2ee23b-85ca-444f-9e1e-2b6f1f46761e service neutron] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:34:38.051680 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1142/2282] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:34:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:34:42.045951 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937a8ad2-bedb-4d2f-803f-2346616014d9 service nova] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:34:42.059286 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937a8ad2-bedb-4d2f-803f-2346616014d9 service nova] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:34:42.065398 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f55367-cafb-45a9-9dba-bac28e94bca5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abe9d8c9291845dfa4ef55034e5c9e32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:34:42.066383 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f55367-cafb-45a9-9dba-bac28e94bca5 service placement] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:34:42.066649 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f55367-cafb-45a9-9dba-bac28e94bca5 service placement] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:34:42.067106 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f55367-cafb-45a9-9dba-bac28e94bca5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:34:42.069260 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-82f55367-cafb-45a9-9dba-bac28e94bca5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:34:42.081644 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f55367-cafb-45a9-9dba-bac28e94bca5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:34:42.083322 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f55367-cafb-45a9-9dba-bac28e94bca5 service placement] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:34:42.093916 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1141/2283] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:34:42 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:34:46.082894 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2ee23b-85ca-444f-9e1e-2b6f1f46761e service neutron] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:34:46.084629 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-7c2ee23b-85ca-444f-9e1e-2b6f1f46761e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:34:46.101259 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2ee23b-85ca-444f-9e1e-2b6f1f46761e service neutron] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:34:46.106498 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057e8a54-d37e-465a-9f95-4224df78b00e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abe9d8c9291845dfa4ef55034e5c9e32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:34:46.107331 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057e8a54-d37e-465a-9f95-4224df78b00e service placement] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:34:46.107611 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057e8a54-d37e-465a-9f95-4224df78b00e service placement] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:34:46.107880 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057e8a54-d37e-465a-9f95-4224df78b00e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:34:46.110359 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-057e8a54-d37e-465a-9f95-4224df78b00e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:34:46.125549 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057e8a54-d37e-465a-9f95-4224df78b00e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:34:46.127498 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057e8a54-d37e-465a-9f95-4224df78b00e service placement] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:34:46.138222 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1143/2284] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:34:46 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:34:46.217517 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f55367-cafb-45a9-9dba-bac28e94bca5 service placement] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:34:46.230036 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f55367-cafb-45a9-9dba-bac28e94bca5 service placement] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:34:46.234774 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4ae43b-b7f6-43fc-9d29-75a55fdcedfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:34:46.235356 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4ae43b-b7f6-43fc-9d29-75a55fdcedfe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:34:46.235639 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4ae43b-b7f6-43fc-9d29-75a55fdcedfe admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:34:46.235917 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4ae43b-b7f6-43fc-9d29-75a55fdcedfe admin admin] PATH_INFO: `/v3/users/27d9cb186f8f40498141aa20db23e9d0` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:34:46.237113 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4ae43b-b7f6-43fc-9d29-75a55fdcedfe admin admin] RBAC: Authorizing `identity:delete_user(user_id=27d9cb186f8f40498141aa20db23e9d0)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:34:46.239037 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4ae43b-b7f6-43fc-9d29-75a55fdcedfe admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:34:46.423846 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-3a4ae43b-b7f6-43fc-9d29-75a55fdcedfe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '27d9cb186f8f40498141aa20db23e9d0'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:34:46.431226 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-3a4ae43b-b7f6-43fc-9d29-75a55fdcedfe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '27d9cb186f8f40498141aa20db23e9d0'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:34:46.431903 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-3a4ae43b-b7f6-43fc-9d29-75a55fdcedfe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '27d9cb186f8f40498141aa20db23e9d0'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:34:46.493685 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-3a4ae43b-b7f6-43fc-9d29-75a55fdcedfe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '27d9cb186f8f40498141aa20db23e9d0'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:34:46.508146 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-3a4ae43b-b7f6-43fc-9d29-75a55fdcedfe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '27d9cb186f8f40498141aa20db23e9d0'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:34:46.514144 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1142/2285] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Aug 24 06:34:46 2022] DELETE /identity/v3/users/27d9cb186f8f40498141aa20db23e9d0 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:34:46.765675 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057e8a54-d37e-465a-9f95-4224df78b00e service placement] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:34:46.778003 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057e8a54-d37e-465a-9f95-4224df78b00e service placement] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:34:46.844419 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b629eaa2-f9c7-439c-9947-824bc3709290 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:34:46.845082 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b629eaa2-f9c7-439c-9947-824bc3709290 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:34:46.845358 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b629eaa2-f9c7-439c-9947-824bc3709290 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:34:46.845632 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b629eaa2-f9c7-439c-9947-824bc3709290 admin admin] PATH_INFO: `/v3/projects/df659b3447644e7dbe0f124b881e4344` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:34:46.846738 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b629eaa2-f9c7-439c-9947-824bc3709290 admin admin] RBAC: Authorizing `identity:delete_project(project_id=df659b3447644e7dbe0f124b881e4344)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:34:46.848740 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b629eaa2-f9c7-439c-9947-824bc3709290 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:34:46.951566 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-b629eaa2-f9c7-439c-9947-824bc3709290 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'df659b3447644e7dbe0f124b881e4344', 'request_id': 'req-b629eaa2-f9c7-439c-9947-824bc3709290'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:34:46.957648 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-b629eaa2-f9c7-439c-9947-824bc3709290 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'df659b3447644e7dbe0f124b881e4344', 'request_id': 'req-b629eaa2-f9c7-439c-9947-824bc3709290'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:34:46.964642 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-b629eaa2-f9c7-439c-9947-824bc3709290 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'df659b3447644e7dbe0f124b881e4344', 'request_id': 'req-b629eaa2-f9c7-439c-9947-824bc3709290'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:34:47.043877 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-b629eaa2-f9c7-439c-9947-824bc3709290 admin admin] The token cache is being invalidate because project df659b3447644e7dbe0f124b881e4344 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81851) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:34:47.044167 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-b629eaa2-f9c7-439c-9947-824bc3709290 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:34:47.044860 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-b629eaa2-f9c7-439c-9947-824bc3709290 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:34:47.046374 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1144/2286] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Aug 24 06:34:46 2022] DELETE /identity/v3/projects/df659b3447644e7dbe0f124b881e4344 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:34:47.050436 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4ae43b-b7f6-43fc-9d29-75a55fdcedfe admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:34:47.051744 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-3a4ae43b-b7f6-43fc-9d29-75a55fdcedfe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:34:47.065366 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4ae43b-b7f6-43fc-9d29-75a55fdcedfe admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:34:47.126488 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5830e94f-f83a-40d9-89c2-09e25b6ee8ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:34:47.127139 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5830e94f-f83a-40d9-89c2-09e25b6ee8ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:34:47.127424 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5830e94f-f83a-40d9-89c2-09e25b6ee8ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:34:47.127703 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5830e94f-f83a-40d9-89c2-09e25b6ee8ad admin admin] PATH_INFO: `/v3/users/7fb72e8309fa4d8197e1f592289b05be` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:34:47.128901 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5830e94f-f83a-40d9-89c2-09e25b6ee8ad admin admin] RBAC: Authorizing `identity:delete_user(user_id=7fb72e8309fa4d8197e1f592289b05be)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:34:47.130868 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5830e94f-f83a-40d9-89c2-09e25b6ee8ad admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:34:47.314101 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-5830e94f-f83a-40d9-89c2-09e25b6ee8ad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7fb72e8309fa4d8197e1f592289b05be'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:34:47.321801 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-5830e94f-f83a-40d9-89c2-09e25b6ee8ad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7fb72e8309fa4d8197e1f592289b05be'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:34:47.322515 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-5830e94f-f83a-40d9-89c2-09e25b6ee8ad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7fb72e8309fa4d8197e1f592289b05be'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:34:47.328392 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=80,keystone:DELETE=18,keystone:INSERT=2 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:34:47.413654 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-5830e94f-f83a-40d9-89c2-09e25b6ee8ad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7fb72e8309fa4d8197e1f592289b05be'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:34:47.425984 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-5830e94f-f83a-40d9-89c2-09e25b6ee8ad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7fb72e8309fa4d8197e1f592289b05be'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:34:47.431933 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1143/2287] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Aug 24 06:34:47 2022] DELETE /identity/v3/users/7fb72e8309fa4d8197e1f592289b05be => generated 0 bytes in 382 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:34:47.680214 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b629eaa2-f9c7-439c-9947-824bc3709290 admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:34:47.688451 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-b629eaa2-f9c7-439c-9947-824bc3709290 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:34:47.696918 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b629eaa2-f9c7-439c-9947-824bc3709290 admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:34:47.705728 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:34:47.757036 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f8de78-50c5-43ad-8365-7f22d5a661d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:34:47.757636 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f8de78-50c5-43ad-8365-7f22d5a661d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:34:47.757916 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f8de78-50c5-43ad-8365-7f22d5a661d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:34:47.758189 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f8de78-50c5-43ad-8365-7f22d5a661d3 admin admin] PATH_INFO: `/v3/projects/d77ef3633caf42679fee9233665da60c` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:34:47.759306 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f8de78-50c5-43ad-8365-7f22d5a661d3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d77ef3633caf42679fee9233665da60c)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:34:47.761263 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f8de78-50c5-43ad-8365-7f22d5a661d3 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:34:47.856094 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-58f8de78-50c5-43ad-8365-7f22d5a661d3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd77ef3633caf42679fee9233665da60c', 'request_id': 'req-58f8de78-50c5-43ad-8365-7f22d5a661d3'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:34:47.861828 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-58f8de78-50c5-43ad-8365-7f22d5a661d3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd77ef3633caf42679fee9233665da60c', 'request_id': 'req-58f8de78-50c5-43ad-8365-7f22d5a661d3'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:34:47.869069 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-58f8de78-50c5-43ad-8365-7f22d5a661d3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd77ef3633caf42679fee9233665da60c', 'request_id': 'req-58f8de78-50c5-43ad-8365-7f22d5a661d3'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:34:47.941218 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-58f8de78-50c5-43ad-8365-7f22d5a661d3 admin admin] The token cache is being invalidate because project d77ef3633caf42679fee9233665da60c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81851) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:34:47.941666 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-58f8de78-50c5-43ad-8365-7f22d5a661d3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:34:47.943293 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-58f8de78-50c5-43ad-8365-7f22d5a661d3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:34:47.954748 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1145/2288] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Aug 24 06:34:47 2022] DELETE /identity/v3/projects/d77ef3633caf42679fee9233665da60c => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:34:57.815001 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=8,keystone:DELETE=2 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:34:58.037400 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=46,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:35:44.956835 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5830e94f-f83a-40d9-89c2-09e25b6ee8ad admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:35:44.958249 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-5830e94f-f83a-40d9-89c2-09e25b6ee8ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:35:45.085151 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5830e94f-f83a-40d9-89c2-09e25b6ee8ad admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:35:45.145350 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5eba445-631e-44bb-8224-0d211c3e0a98 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abe9d8c9291845dfa4ef55034e5c9e32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:35:45.145939 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5eba445-631e-44bb-8224-0d211c3e0a98 service placement] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:35:45.146212 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5eba445-631e-44bb-8224-0d211c3e0a98 service placement] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:35:45.146478 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5eba445-631e-44bb-8224-0d211c3e0a98 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:35:45.148187 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-c5eba445-631e-44bb-8224-0d211c3e0a98 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:35:45.159164 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5eba445-631e-44bb-8224-0d211c3e0a98 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:35:45.160788 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5eba445-631e-44bb-8224-0d211c3e0a98 service placement] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:35:45.170229 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1144/2289] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:35:44 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:35:55.166246 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=84 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:36:35.359256 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f8de78-50c5-43ad-8365-7f22d5a661d3 admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:36:35.360767 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-58f8de78-50c5-43ad-8365-7f22d5a661d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:36:35.430619 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f8de78-50c5-43ad-8365-7f22d5a661d3 admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:36:35.434467 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d54f380-9f19-49c2-b4da-812c90134644 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bb6c79db75e4d169331c02133d1ec32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:36:35.435071 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d54f380-9f19-49c2-b4da-812c90134644 service neutron] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:36:35.435349 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d54f380-9f19-49c2-b4da-812c90134644 service neutron] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:36:35.435614 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d54f380-9f19-49c2-b4da-812c90134644 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:36:35.437353 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-3d54f380-9f19-49c2-b4da-812c90134644 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:36:35.514035 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d54f380-9f19-49c2-b4da-812c90134644 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:36:35.515382 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d54f380-9f19-49c2-b4da-812c90134644 service neutron] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:36:35.535853 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1146/2290] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:36:35 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:36:37.646068 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5eba445-631e-44bb-8224-0d211c3e0a98 service placement] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:36:37.647352 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-c5eba445-631e-44bb-8224-0d211c3e0a98 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:36:37.778172 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5eba445-631e-44bb-8224-0d211c3e0a98 service placement] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:36:37.783102 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ddc685-ca10-4ad3-8828-162d12efaf2b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c4def55ac0e48b29836bc5c64bc7402', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:36:37.784042 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ddc685-ca10-4ad3-8828-162d12efaf2b service cinder] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:36:37.784515 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ddc685-ca10-4ad3-8828-162d12efaf2b service cinder] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:36:37.784971 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ddc685-ca10-4ad3-8828-162d12efaf2b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:36:37.787149 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-b3ddc685-ca10-4ad3-8828-162d12efaf2b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:36:37.892363 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ddc685-ca10-4ad3-8828-162d12efaf2b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:36:37.894997 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ddc685-ca10-4ad3-8828-162d12efaf2b service cinder] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:36:37.906648 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1145/2291] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:36:37 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 263 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:36:40.355297 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d54f380-9f19-49c2-b4da-812c90134644 service neutron] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:36:40.357024 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-3d54f380-9f19-49c2-b4da-812c90134644 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:36:40.386172 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d54f380-9f19-49c2-b4da-812c90134644 service neutron] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:36:40.392104 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377178f0-5616-477f-bfa0-cd55e563e18b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb6d3743340241f98cacb9e0d063298c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:36:40.392789 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377178f0-5616-477f-bfa0-cd55e563e18b service nova] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:36:40.393567 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377178f0-5616-477f-bfa0-cd55e563e18b service nova] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:36:40.393858 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377178f0-5616-477f-bfa0-cd55e563e18b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:36:40.395658 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-377178f0-5616-477f-bfa0-cd55e563e18b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:36:40.469328 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:36:40.475909 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377178f0-5616-477f-bfa0-cd55e563e18b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:36:40.478802 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377178f0-5616-477f-bfa0-cd55e563e18b service nova] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:36:40.512408 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1147/2292] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:36:40 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:36:40.575773 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ddc685-ca10-4ad3-8828-162d12efaf2b service cinder] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:36:40.577431 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-b3ddc685-ca10-4ad3-8828-162d12efaf2b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:36:40.609937 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ddc685-ca10-4ad3-8828-162d12efaf2b service cinder] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:36:40.615894 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab79593-ea08-419d-8bf9-24a31dc33e92 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bb6c79db75e4d169331c02133d1ec32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:36:40.617041 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab79593-ea08-419d-8bf9-24a31dc33e92 service neutron] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:36:40.617461 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab79593-ea08-419d-8bf9-24a31dc33e92 service neutron] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:36:40.617844 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab79593-ea08-419d-8bf9-24a31dc33e92 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:36:40.620315 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-9ab79593-ea08-419d-8bf9-24a31dc33e92 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:36:40.637064 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab79593-ea08-419d-8bf9-24a31dc33e92 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:36:40.638600 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab79593-ea08-419d-8bf9-24a31dc33e92 service neutron] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:36:40.656178 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1146/2293] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:36:40 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:36:42.249058 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377178f0-5616-477f-bfa0-cd55e563e18b service nova] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:36:42.261684 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377178f0-5616-477f-bfa0-cd55e563e18b service nova] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:36:42.267870 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a93dcf-9d7a-4d80-8fa6-bb5f09a28957 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb6d3743340241f98cacb9e0d063298c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:36:42.269308 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a93dcf-9d7a-4d80-8fa6-bb5f09a28957 service nova] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:36:42.280729 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a93dcf-9d7a-4d80-8fa6-bb5f09a28957 service nova] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:36:42.280729 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a93dcf-9d7a-4d80-8fa6-bb5f09a28957 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:36:42.280729 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-c8a93dcf-9d7a-4d80-8fa6-bb5f09a28957 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:36:42.339229 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a93dcf-9d7a-4d80-8fa6-bb5f09a28957 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:36:42.340933 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a93dcf-9d7a-4d80-8fa6-bb5f09a28957 service nova] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:36:42.352122 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1148/2294] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:36:42 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:36:50.646143 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=67 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:36:52.347538 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=35 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:37:43.711178 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab79593-ea08-419d-8bf9-24a31dc33e92 service neutron] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:37:43.721502 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab79593-ea08-419d-8bf9-24a31dc33e92 service neutron] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:37:43.725747 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c217160-8179-4754-b87b-b340b444ea62 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bb6c79db75e4d169331c02133d1ec32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:37:43.726470 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c217160-8179-4754-b87b-b340b444ea62 service neutron] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:37:43.726840 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c217160-8179-4754-b87b-b340b444ea62 service neutron] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:37:43.727217 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c217160-8179-4754-b87b-b340b444ea62 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:37:43.729133 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-8c217160-8179-4754-b87b-b340b444ea62 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:37:43.754837 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c217160-8179-4754-b87b-b340b444ea62 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:37:43.756186 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c217160-8179-4754-b87b-b340b444ea62 service neutron] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:37:43.765161 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1147/2295] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:37:43 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:37:53.760901 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=11 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:38:38.221796 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a93dcf-9d7a-4d80-8fa6-bb5f09a28957 service nova] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:38:38.232098 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a93dcf-9d7a-4d80-8fa6-bb5f09a28957 service nova] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:38:38.235921 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422161a9-7f7c-4756-8a05-eca8ae456598 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c4def55ac0e48b29836bc5c64bc7402', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:38:38.236560 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422161a9-7f7c-4756-8a05-eca8ae456598 service cinder] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:38:38.236844 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422161a9-7f7c-4756-8a05-eca8ae456598 service cinder] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:38:38.237112 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422161a9-7f7c-4756-8a05-eca8ae456598 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:38:38.239090 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-422161a9-7f7c-4756-8a05-eca8ae456598 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:38:38.331234 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422161a9-7f7c-4756-8a05-eca8ae456598 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:38:38.332695 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422161a9-7f7c-4756-8a05-eca8ae456598 service cinder] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:38:38.354645 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1149/2296] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:38:38 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 06:38:38.501491 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c217160-8179-4754-b87b-b340b444ea62 service neutron] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:38:38.515064 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c217160-8179-4754-b87b-b340b444ea62 service neutron] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:38:38.518956 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4364ee98-cafc-40cf-a7b2-5e4bf70652ba service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c4def55ac0e48b29836bc5c64bc7402', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:38:38.519637 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4364ee98-cafc-40cf-a7b2-5e4bf70652ba service cinder] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:38:38.519952 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4364ee98-cafc-40cf-a7b2-5e4bf70652ba service cinder] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:38:38.520272 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4364ee98-cafc-40cf-a7b2-5e4bf70652ba service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:38:38.522389 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-4364ee98-cafc-40cf-a7b2-5e4bf70652ba service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:38:38.656749 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4364ee98-cafc-40cf-a7b2-5e4bf70652ba service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:38:38.658446 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4364ee98-cafc-40cf-a7b2-5e4bf70652ba service cinder] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:38:38.680574 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1148/2297] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:38:38 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 06:38:40.578577 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5ea365-56c2-4763-bf5f-975a9ff9f0c1 None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:38:40.579217 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5ea365-56c2-4763-bf5f-975a9ff9f0c1 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:38:40.579217 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5ea365-56c2-4763-bf5f-975a9ff9f0c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:38:40.613434 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-3d5ea365-56c2-4763-bf5f-975a9ff9f0c1 None None] MFA Rules not processed for user `6abfd0a28f9e482badae7018f1b7f9a1`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:38:40.615315 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-3d5ea365-56c2-4763-bf5f-975a9ff9f0c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:38:40.623511 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1150/2298] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:38:40 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:38:40.997326 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4364ee98-cafc-40cf-a7b2-5e4bf70652ba service cinder] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:38:41.007811 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4364ee98-cafc-40cf-a7b2-5e4bf70652ba service cinder] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:38:41.011484 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdaa706a-aa8c-49a9-b97b-5f5609910352 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bb6c79db75e4d169331c02133d1ec32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:38:41.012103 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdaa706a-aa8c-49a9-b97b-5f5609910352 service neutron] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:38:41.012379 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdaa706a-aa8c-49a9-b97b-5f5609910352 service neutron] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:38:41.012691 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdaa706a-aa8c-49a9-b97b-5f5609910352 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:38:41.015079 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-bdaa706a-aa8c-49a9-b97b-5f5609910352 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:38:41.026107 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdaa706a-aa8c-49a9-b97b-5f5609910352 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:38:41.027566 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdaa706a-aa8c-49a9-b97b-5f5609910352 service neutron] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:38:41.037207 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1149/2299] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:38:40 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 06:38:41.506331 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5ea365-56c2-4763-bf5f-975a9ff9f0c1 None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:38:41.508674 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-3d5ea365-56c2-4763-bf5f-975a9ff9f0c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:38:41.536589 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5ea365-56c2-4763-bf5f-975a9ff9f0c1 None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:38:41.543446 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d5739c-7a07-4d9c-b4de-373572155031 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abe9d8c9291845dfa4ef55034e5c9e32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:38:41.544328 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d5739c-7a07-4d9c-b4de-373572155031 service placement] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:38:41.544750 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d5739c-7a07-4d9c-b4de-373572155031 service placement] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:38:41.545107 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d5739c-7a07-4d9c-b4de-373572155031 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:38:41.547945 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-76d5739c-7a07-4d9c-b4de-373572155031 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:38:41.566996 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d5739c-7a07-4d9c-b4de-373572155031 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:38:41.570247 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d5739c-7a07-4d9c-b4de-373572155031 service placement] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:38:41.588011 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1151/2300] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:38:41 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:38:46.037594 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdaa706a-aa8c-49a9-b97b-5f5609910352 service neutron] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:38:46.053201 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdaa706a-aa8c-49a9-b97b-5f5609910352 service neutron] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:38:46.066739 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1878f0ef-0890-4abc-bbbc-85aba0b77ad3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb6d3743340241f98cacb9e0d063298c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:38:46.067832 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1878f0ef-0890-4abc-bbbc-85aba0b77ad3 service nova] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:38:46.068320 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1878f0ef-0890-4abc-bbbc-85aba0b77ad3 service nova] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:38:46.068854 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1878f0ef-0890-4abc-bbbc-85aba0b77ad3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:38:46.071911 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-1878f0ef-0890-4abc-bbbc-85aba0b77ad3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:38:46.087270 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1878f0ef-0890-4abc-bbbc-85aba0b77ad3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:38:46.089907 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1878f0ef-0890-4abc-bbbc-85aba0b77ad3 service nova] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:38:46.101883 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1150/2301] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:38:46 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:38:50.033356 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d5739c-7a07-4d9c-b4de-373572155031 service placement] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:38:50.047550 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d5739c-7a07-4d9c-b4de-373572155031 service placement] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:38:50.053796 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b694d5e4-cf27-4dab-9ab4-93c788d19bce service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb6d3743340241f98cacb9e0d063298c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:38:50.054517 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b694d5e4-cf27-4dab-9ab4-93c788d19bce service nova] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:38:50.054705 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b694d5e4-cf27-4dab-9ab4-93c788d19bce service nova] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:38:50.054981 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b694d5e4-cf27-4dab-9ab4-93c788d19bce service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:38:50.056812 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-b694d5e4-cf27-4dab-9ab4-93c788d19bce service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:38:50.070763 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b694d5e4-cf27-4dab-9ab4-93c788d19bce service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:38:50.072378 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b694d5e4-cf27-4dab-9ab4-93c788d19bce service nova] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:38:50.085035 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1152/2302] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:38:50 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:38:56.097754 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=53 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:39:00.079739 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=59 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:39:37.787890 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ece6d5-cda6-4625-9346-101c716f7ed3 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:39:37.788431 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ece6d5-cda6-4625-9346-101c716f7ed3 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:39:37.788431 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ece6d5-cda6-4625-9346-101c716f7ed3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:39:37.820700 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-f9ece6d5-cda6-4625-9346-101c716f7ed3 None None] MFA Rules not processed for user `5059723f417445fa934ecfe3e50d0197`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:39:37.821911 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-f9ece6d5-cda6-4625-9346-101c716f7ed3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:39:37.829229 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1151/2303] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:39:37 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:39:47.818390 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=6 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:40:50.982531 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b694d5e4-cf27-4dab-9ab4-93c788d19bce service nova] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:40:50.994650 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b694d5e4-cf27-4dab-9ab4-93c788d19bce service nova] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:40:51.000091 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abb50c7-4cf2-441f-95fc-b4ff8c5c908a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abe9d8c9291845dfa4ef55034e5c9e32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:40:51.000756 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abb50c7-4cf2-441f-95fc-b4ff8c5c908a service placement] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:40:51.001036 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abb50c7-4cf2-441f-95fc-b4ff8c5c908a service placement] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:40:51.001308 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abb50c7-4cf2-441f-95fc-b4ff8c5c908a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:40:51.010986 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abb50c7-4cf2-441f-95fc-b4ff8c5c908a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:40:51.012623 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abb50c7-4cf2-441f-95fc-b4ff8c5c908a service placement] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:40:51.023846 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1153/2304] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:40:50 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:41:01.018898 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=6 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:41:35.679165 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ece6d5-cda6-4625-9346-101c716f7ed3 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:41:35.689476 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ece6d5-cda6-4625-9346-101c716f7ed3 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:35.693265 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6d7903-03af-4b82-b824-4fe2baf2e3fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bb6c79db75e4d169331c02133d1ec32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:35.693879 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6d7903-03af-4b82-b824-4fe2baf2e3fd service neutron] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:35.694158 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6d7903-03af-4b82-b824-4fe2baf2e3fd service neutron] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:35.694436 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6d7903-03af-4b82-b824-4fe2baf2e3fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:35.696230 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-5f6d7903-03af-4b82-b824-4fe2baf2e3fd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:41:35.790137 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6d7903-03af-4b82-b824-4fe2baf2e3fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:35.791589 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6d7903-03af-4b82-b824-4fe2baf2e3fd service neutron] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:35.811171 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1152/2305] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:41:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:41:39.728707 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abb50c7-4cf2-441f-95fc-b4ff8c5c908a service placement] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:41:39.730195 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-9abb50c7-4cf2-441f-95fc-b4ff8c5c908a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:41:39.744943 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abb50c7-4cf2-441f-95fc-b4ff8c5c908a service placement] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:39.750321 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191d71a1-0847-4bd4-b7c8-42f7b2eef82c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abe9d8c9291845dfa4ef55034e5c9e32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:39.750929 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191d71a1-0847-4bd4-b7c8-42f7b2eef82c service placement] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:39.751207 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191d71a1-0847-4bd4-b7c8-42f7b2eef82c service placement] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:39.751480 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191d71a1-0847-4bd4-b7c8-42f7b2eef82c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:39.753319 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-191d71a1-0847-4bd4-b7c8-42f7b2eef82c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:41:39.766196 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191d71a1-0847-4bd4-b7c8-42f7b2eef82c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:39.767794 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191d71a1-0847-4bd4-b7c8-42f7b2eef82c service placement] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:39.777863 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1154/2306] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:41:39 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:41:39.858379 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6d7903-03af-4b82-b824-4fe2baf2e3fd service neutron] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:41:39.860827 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-5f6d7903-03af-4b82-b824-4fe2baf2e3fd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:41:39.884074 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6d7903-03af-4b82-b824-4fe2baf2e3fd service neutron] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:39.888699 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d89116-0b59-4092-a46d-798c235add69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:39.889332 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d89116-0b59-4092-a46d-798c235add69 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:39.889612 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d89116-0b59-4092-a46d-798c235add69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:39.889885 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d89116-0b59-4092-a46d-798c235add69 admin admin] PATH_INFO: `/v3/users/25908fb02f7f46f5b9e483811302a67e` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:39.904319 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d89116-0b59-4092-a46d-798c235add69 admin admin] RBAC: Authorizing `identity:delete_user(user_id=25908fb02f7f46f5b9e483811302a67e)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:39.906405 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d89116-0b59-4092-a46d-798c235add69 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:40.139640 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-99d89116-0b59-4092-a46d-798c235add69 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '25908fb02f7f46f5b9e483811302a67e'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:40.147714 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-99d89116-0b59-4092-a46d-798c235add69 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '25908fb02f7f46f5b9e483811302a67e'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:40.148483 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-99d89116-0b59-4092-a46d-798c235add69 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '25908fb02f7f46f5b9e483811302a67e'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:40.222331 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-99d89116-0b59-4092-a46d-798c235add69 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '25908fb02f7f46f5b9e483811302a67e'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:40.236659 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-99d89116-0b59-4092-a46d-798c235add69 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '25908fb02f7f46f5b9e483811302a67e'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:40.243942 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1153/2307] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Aug 24 06:41:39 2022] DELETE /identity/v3/users/25908fb02f7f46f5b9e483811302a67e => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:41:40.499073 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191d71a1-0847-4bd4-b7c8-42f7b2eef82c service placement] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:41:40.510223 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191d71a1-0847-4bd4-b7c8-42f7b2eef82c service placement] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:40.570560 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424fa445-380e-4f6f-82d3-9e0bcf51db6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:40.571217 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424fa445-380e-4f6f-82d3-9e0bcf51db6b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:40.571498 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424fa445-380e-4f6f-82d3-9e0bcf51db6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:40.571773 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424fa445-380e-4f6f-82d3-9e0bcf51db6b admin admin] PATH_INFO: `/v3/projects/ff194aa4a2d744caaa09e0ceee54b1d2` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:40.582609 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424fa445-380e-4f6f-82d3-9e0bcf51db6b admin admin] RBAC: Authorizing `identity:delete_project(project_id=ff194aa4a2d744caaa09e0ceee54b1d2)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:40.585154 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424fa445-380e-4f6f-82d3-9e0bcf51db6b admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:40.716877 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-424fa445-380e-4f6f-82d3-9e0bcf51db6b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ff194aa4a2d744caaa09e0ceee54b1d2', 'request_id': 'req-424fa445-380e-4f6f-82d3-9e0bcf51db6b'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:40.722680 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-424fa445-380e-4f6f-82d3-9e0bcf51db6b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ff194aa4a2d744caaa09e0ceee54b1d2', 'request_id': 'req-424fa445-380e-4f6f-82d3-9e0bcf51db6b'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:40.850515 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-424fa445-380e-4f6f-82d3-9e0bcf51db6b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ff194aa4a2d744caaa09e0ceee54b1d2', 'request_id': 'req-424fa445-380e-4f6f-82d3-9e0bcf51db6b'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:40.936147 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-424fa445-380e-4f6f-82d3-9e0bcf51db6b admin admin] The token cache is being invalidate because project ff194aa4a2d744caaa09e0ceee54b1d2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81851) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:41:40.936458 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-424fa445-380e-4f6f-82d3-9e0bcf51db6b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:40.937108 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-424fa445-380e-4f6f-82d3-9e0bcf51db6b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:40.938610 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1155/2308] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Aug 24 06:41:40 2022] DELETE /identity/v3/projects/ff194aa4a2d744caaa09e0ceee54b1d2 => generated 0 bytes in 440 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:41:40.945094 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fed5623-a5fe-4410-9ab8-d6c4aa27e066 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:40.945668 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fed5623-a5fe-4410-9ab8-d6c4aa27e066 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:40.945783 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fed5623-a5fe-4410-9ab8-d6c4aa27e066 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:40.984672 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:41:41.005531 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-6fed5623-a5fe-4410-9ab8-d6c4aa27e066 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:41:41.006770 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-6fed5623-a5fe-4410-9ab8-d6c4aa27e066 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:41:41.067972 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1154/2309] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:41:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:41:41.076126 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7cfc19-a711-4e3d-bc8c-3e78874b2e52 None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:41.076408 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7cfc19-a711-4e3d-bc8c-3e78874b2e52 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:41.076710 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7cfc19-a711-4e3d-bc8c-3e78874b2e52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:41.096010 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-8a7cfc19-a711-4e3d-bc8c-3e78874b2e52 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:41:41.097222 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-8a7cfc19-a711-4e3d-bc8c-3e78874b2e52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:41:41.105078 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1156/2310] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:41:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:41:41.109421 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fed5623-a5fe-4410-9ab8-d6c4aa27e066 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:41:41.118916 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fed5623-a5fe-4410-9ab8-d6c4aa27e066 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:41.123381 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9595d3-a184-4ca8-8448-c49cc0431e10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:41.123980 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9595d3-a184-4ca8-8448-c49cc0431e10 admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:41.124257 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9595d3-a184-4ca8-8448-c49cc0431e10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:41.124629 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9595d3-a184-4ca8-8448-c49cc0431e10 admin admin] PATH_INFO: `/v3/domains` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:41.125274 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9595d3-a184-4ca8-8448-c49cc0431e10 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:41:41.125570 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9595d3-a184-4ca8-8448-c49cc0431e10 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:41.127155 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9595d3-a184-4ca8-8448-c49cc0431e10 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:41.139491 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1155/2311] 10.222.0.62 () {58 vars in 1153 bytes} [Wed Aug 24 06:41:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:41:41.143870 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7cfc19-a711-4e3d-bc8c-3e78874b2e52 None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:41:41.153754 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7cfc19-a711-4e3d-bc8c-3e78874b2e52 None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:41.158318 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7b9110-af06-44ce-b302-a2d0dd16cd20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:41.158914 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7b9110-af06-44ce-b302-a2d0dd16cd20 admin admin] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:41.159191 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7b9110-af06-44ce-b302-a2d0dd16cd20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:41.159461 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7b9110-af06-44ce-b302-a2d0dd16cd20 admin admin] PATH_INFO: `/v3/projects` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:41.160071 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7b9110-af06-44ce-b302-a2d0dd16cd20 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:41.162148 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7b9110-af06-44ce-b302-a2d0dd16cd20 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:41.292026 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1157/2312] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Aug 24 06:41:41 2022] POST /identity/v3/projects => generated 371 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:41:41.296251 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9595d3-a184-4ca8-8448-c49cc0431e10 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:41:41.306821 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9595d3-a184-4ca8-8448-c49cc0431e10 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:41.362027 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c709c5bd-d825-47c7-838c-3a79fca14708 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:41.362544 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c709c5bd-d825-47c7-838c-3a79fca14708 admin admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:41.362821 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c709c5bd-d825-47c7-838c-3a79fca14708 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:41.363092 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c709c5bd-d825-47c7-838c-3a79fca14708 admin admin] PATH_INFO: `/v3/users` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:41.363718 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c709c5bd-d825-47c7-838c-3a79fca14708 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:41.366179 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c709c5bd-d825-47c7-838c-3a79fca14708 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:41.485231 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1156/2313] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Aug 24 06:41:41 2022] POST /identity/v3/users => generated 286 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:41:41.490219 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7b9110-af06-44ce-b302-a2d0dd16cd20 admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:41:41.500843 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7b9110-af06-44ce-b302-a2d0dd16cd20 admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:41.506522 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c35d9d-d375-4706-8c88-a9edf28e0ac2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:41.507129 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c35d9d-d375-4706-8c88-a9edf28e0ac2 admin admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:41.507408 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c35d9d-d375-4706-8c88-a9edf28e0ac2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:41.507679 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c35d9d-d375-4706-8c88-a9edf28e0ac2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:41.508320 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c35d9d-d375-4706-8c88-a9edf28e0ac2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:41.509949 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c35d9d-d375-4706-8c88-a9edf28e0ac2 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:41.520107 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1158/2314] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Aug 24 06:41:41 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:41:41.524252 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c709c5bd-d825-47c7-838c-3a79fca14708 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:41:41.534620 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c709c5bd-d825-47c7-838c-3a79fca14708 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:41.539500 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09eba511-80cb-4112-ab4f-3e5504708442 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:41.540111 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09eba511-80cb-4112-ab4f-3e5504708442 admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:41.540395 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09eba511-80cb-4112-ab4f-3e5504708442 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:41.540705 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09eba511-80cb-4112-ab4f-3e5504708442 admin admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:41.541376 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09eba511-80cb-4112-ab4f-3e5504708442 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:41.543081 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09eba511-80cb-4112-ab4f-3e5504708442 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:41.554652 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1157/2315] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Aug 24 06:41:41 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:41:41.558885 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c35d9d-d375-4706-8c88-a9edf28e0ac2 admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:41:41.569761 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c35d9d-d375-4706-8c88-a9edf28e0ac2 admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:41.574263 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec3d3b4-d49f-4a86-b1d6-204b45d52f89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:41.574902 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec3d3b4-d49f-4a86-b1d6-204b45d52f89 admin admin] REQUEST_METHOD: `PUT` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:41.575182 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec3d3b4-d49f-4a86-b1d6-204b45d52f89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:41.575456 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec3d3b4-d49f-4a86-b1d6-204b45d52f89 admin admin] PATH_INFO: `/v3/projects/89e7de2d11ec4e86b293a92de4e106b5/users/c8f4564533c846b49d46fc26da876c9b/roles/580aa7dbb38d4cff8f9d1bb8983f99ef` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:41.590162 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec3d3b4-d49f-4a86-b1d6-204b45d52f89 admin admin] RBAC: Authorizing `identity:create_grant(project_id=89e7de2d11ec4e86b293a92de4e106b5, user_id=c8f4564533c846b49d46fc26da876c9b, role_id=580aa7dbb38d4cff8f9d1bb8983f99ef)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:41.592821 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec3d3b4-d49f-4a86-b1d6-204b45d52f89 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:41.596051 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:41:41.702645 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1159/2316] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Aug 24 06:41:41 2022] PUT /identity/v3/projects/89e7de2d11ec4e86b293a92de4e106b5/users/c8f4564533c846b49d46fc26da876c9b/roles/580aa7dbb38d4cff8f9d1bb8983f99ef => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:41:41.716761 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7588f470-ca1d-46e2-aa24-e788fd1572b9 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:41.717047 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7588f470-ca1d-46e2-aa24-e788fd1572b9 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:41.717319 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7588f470-ca1d-46e2-aa24-e788fd1572b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:41.748824 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-7588f470-ca1d-46e2-aa24-e788fd1572b9 None None] MFA Rules not processed for user `c8f4564533c846b49d46fc26da876c9b`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:41:41.754169 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-7588f470-ca1d-46e2-aa24-e788fd1572b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:41:41.798263 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:41:41.815479 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1158/2317] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:41:41 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 06:41:41.820253 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec3d3b4-d49f-4a86-b1d6-204b45d52f89 admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:41:41.882529 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec3d3b4-d49f-4a86-b1d6-204b45d52f89 admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:41.887303 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719bd99a-b758-4021-a84b-75f018318c34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:41.887904 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719bd99a-b758-4021-a84b-75f018318c34 admin admin] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:41.888181 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719bd99a-b758-4021-a84b-75f018318c34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:41.888451 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719bd99a-b758-4021-a84b-75f018318c34 admin admin] PATH_INFO: `/v3/projects` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:41.889099 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719bd99a-b758-4021-a84b-75f018318c34 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:41.890989 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719bd99a-b758-4021-a84b-75f018318c34 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:41.978610 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1160/2318] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Aug 24 06:41:41 2022] POST /identity/v3/projects => generated 373 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:41:41.982652 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7588f470-ca1d-46e2-aa24-e788fd1572b9 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:41:41.993996 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7588f470-ca1d-46e2-aa24-e788fd1572b9 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:42.051353 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ff77b7-1950-4c43-8fe5-5f6799f88b5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:42.051985 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ff77b7-1950-4c43-8fe5-5f6799f88b5e admin admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:42.052263 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ff77b7-1950-4c43-8fe5-5f6799f88b5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:42.052564 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ff77b7-1950-4c43-8fe5-5f6799f88b5e admin admin] PATH_INFO: `/v3/users` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:42.053201 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ff77b7-1950-4c43-8fe5-5f6799f88b5e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:42.055142 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ff77b7-1950-4c43-8fe5-5f6799f88b5e admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:42.136845 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1159/2319] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Aug 24 06:41:41 2022] POST /identity/v3/users => generated 293 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:41:42.140998 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719bd99a-b758-4021-a84b-75f018318c34 admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:41:42.152716 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719bd99a-b758-4021-a84b-75f018318c34 admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:42.157414 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a534bdb8-7c28-47c2-8808-03e0616afdc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:42.158052 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a534bdb8-7c28-47c2-8808-03e0616afdc3 admin admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:42.158329 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a534bdb8-7c28-47c2-8808-03e0616afdc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:42.158598 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a534bdb8-7c28-47c2-8808-03e0616afdc3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:42.159240 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a534bdb8-7c28-47c2-8808-03e0616afdc3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:42.160939 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a534bdb8-7c28-47c2-8808-03e0616afdc3 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:42.172148 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1161/2320] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Aug 24 06:41:42 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:41:42.176400 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ff77b7-1950-4c43-8fe5-5f6799f88b5e admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:41:42.187571 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ff77b7-1950-4c43-8fe5-5f6799f88b5e admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:42.192523 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286aceb8-520a-4fd5-9de0-f4c3711dfe4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:42.194917 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286aceb8-520a-4fd5-9de0-f4c3711dfe4b admin admin] REQUEST_METHOD: `PUT` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:42.194917 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286aceb8-520a-4fd5-9de0-f4c3711dfe4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:42.194917 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286aceb8-520a-4fd5-9de0-f4c3711dfe4b admin admin] PATH_INFO: `/v3/projects/c7704cd752bb48508f5540c94cd42ee6/users/389e6edbab16426788696806b2419165/roles/49e47cd157d743f6905f1db07ec878a7` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:42.209844 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286aceb8-520a-4fd5-9de0-f4c3711dfe4b admin admin] RBAC: Authorizing `identity:create_grant(project_id=c7704cd752bb48508f5540c94cd42ee6, user_id=389e6edbab16426788696806b2419165, role_id=49e47cd157d743f6905f1db07ec878a7)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:42.212673 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286aceb8-520a-4fd5-9de0-f4c3711dfe4b admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:42.308171 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1160/2321] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Aug 24 06:41:42 2022] PUT /identity/v3/projects/c7704cd752bb48508f5540c94cd42ee6/users/389e6edbab16426788696806b2419165/roles/49e47cd157d743f6905f1db07ec878a7 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:41:42.312320 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a534bdb8-7c28-47c2-8808-03e0616afdc3 admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:41:42.322665 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a534bdb8-7c28-47c2-8808-03e0616afdc3 admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:42.379077 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fec3f3a-5bbf-4c9b-aac3-b6462770b55a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:42.380311 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fec3f3a-5bbf-4c9b-aac3-b6462770b55a admin admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:42.380681 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fec3f3a-5bbf-4c9b-aac3-b6462770b55a admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:42.380958 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fec3f3a-5bbf-4c9b-aac3-b6462770b55a admin admin] PATH_INFO: `/v3/roles` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:42.381803 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fec3f3a-5bbf-4c9b-aac3-b6462770b55a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:42.383432 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fec3f3a-5bbf-4c9b-aac3-b6462770b55a admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:42.393368 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1162/2322] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Aug 24 06:41:42 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:41:42.398286 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286aceb8-520a-4fd5-9de0-f4c3711dfe4b admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:41:42.410515 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286aceb8-520a-4fd5-9de0-f4c3711dfe4b admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:42.415432 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1305488f-313e-490f-a470-1aa20161b5ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:42.416087 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1305488f-313e-490f-a470-1aa20161b5ed admin admin] REQUEST_METHOD: `PUT` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:42.416371 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1305488f-313e-490f-a470-1aa20161b5ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:42.416682 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1305488f-313e-490f-a470-1aa20161b5ed admin admin] PATH_INFO: `/v3/projects/c7704cd752bb48508f5540c94cd42ee6/users/389e6edbab16426788696806b2419165/roles/49e47cd157d743f6905f1db07ec878a7` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:42.418579 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1305488f-313e-490f-a470-1aa20161b5ed admin admin] RBAC: Authorizing `identity:create_grant(project_id=c7704cd752bb48508f5540c94cd42ee6, user_id=389e6edbab16426788696806b2419165, role_id=49e47cd157d743f6905f1db07ec878a7)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:42.421283 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1305488f-313e-490f-a470-1aa20161b5ed admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:42.431896 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1161/2323] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Aug 24 06:41:42 2022] PUT /identity/v3/projects/c7704cd752bb48508f5540c94cd42ee6/users/389e6edbab16426788696806b2419165/roles/49e47cd157d743f6905f1db07ec878a7 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:41:42.444593 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e2f3b6-5a9e-4609-b198-7c6237317c10 None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:42.444885 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e2f3b6-5a9e-4609-b198-7c6237317c10 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:42.445164 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e2f3b6-5a9e-4609-b198-7c6237317c10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:42.473251 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-72e2f3b6-5a9e-4609-b198-7c6237317c10 None None] MFA Rules not processed for user `389e6edbab16426788696806b2419165`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:41:42.474517 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-72e2f3b6-5a9e-4609-b198-7c6237317c10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:41:42.531963 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:41:42.550139 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1163/2324] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:41:42 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:41:52.428069 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=48,keystone:INSERT=5 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:41:52.587004 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=4 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:42:40.467586 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1305488f-313e-490f-a470-1aa20161b5ed admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:42:40.469272 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-1305488f-313e-490f-a470-1aa20161b5ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:42:40.539723 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1305488f-313e-490f-a470-1aa20161b5ed admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:42:40.543259 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee26ea5-03bf-4331-b87b-2c457a3c7f61 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c4def55ac0e48b29836bc5c64bc7402', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:42:40.543856 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee26ea5-03bf-4331-b87b-2c457a3c7f61 service cinder] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:42:40.544131 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee26ea5-03bf-4331-b87b-2c457a3c7f61 service cinder] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:42:40.544423 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee26ea5-03bf-4331-b87b-2c457a3c7f61 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:42:40.546152 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-7ee26ea5-03bf-4331-b87b-2c457a3c7f61 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:42:40.616158 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee26ea5-03bf-4331-b87b-2c457a3c7f61 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:42:40.617644 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee26ea5-03bf-4331-b87b-2c457a3c7f61 service cinder] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:42:40.629942 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1162/2325] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:42:40 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:42:40.707551 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e2f3b6-5a9e-4609-b198-7c6237317c10 None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:42:40.727200 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e2f3b6-5a9e-4609-b198-7c6237317c10 None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:42:40.727200 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd0eff6-1e60-4587-bb07-9eed40cb039a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c4def55ac0e48b29836bc5c64bc7402', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:42:40.727200 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd0eff6-1e60-4587-bb07-9eed40cb039a service cinder] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:42:40.727200 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd0eff6-1e60-4587-bb07-9eed40cb039a service cinder] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:42:40.727711 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd0eff6-1e60-4587-bb07-9eed40cb039a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:42:40.727711 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-7dd0eff6-1e60-4587-bb07-9eed40cb039a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:42:40.808139 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd0eff6-1e60-4587-bb07-9eed40cb039a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:42:40.809605 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd0eff6-1e60-4587-bb07-9eed40cb039a service cinder] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:42:40.827643 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1164/2326] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:42:40 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:42:45.667597 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee26ea5-03bf-4331-b87b-2c457a3c7f61 service cinder] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:42:45.669115 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-7ee26ea5-03bf-4331-b87b-2c457a3c7f61 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:42:45.726046 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee26ea5-03bf-4331-b87b-2c457a3c7f61 service cinder] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:42:45.729617 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b38de62-dc86-49e1-92d3-81743cde5395 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bb6c79db75e4d169331c02133d1ec32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:42:45.730339 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b38de62-dc86-49e1-92d3-81743cde5395 service neutron] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:42:45.730618 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b38de62-dc86-49e1-92d3-81743cde5395 service neutron] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:42:45.730886 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b38de62-dc86-49e1-92d3-81743cde5395 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:42:45.732825 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-1b38de62-dc86-49e1-92d3-81743cde5395 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:42:45.742721 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b38de62-dc86-49e1-92d3-81743cde5395 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:42:45.744218 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b38de62-dc86-49e1-92d3-81743cde5395 service neutron] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:42:45.752295 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1163/2327] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:42:45 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:42:50.825132 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=41 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:42:55.748750 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=84 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:43:35.648967 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd0eff6-1e60-4587-bb07-9eed40cb039a service cinder] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:35.650413 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-7dd0eff6-1e60-4587-bb07-9eed40cb039a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:43:35.661187 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd0eff6-1e60-4587-bb07-9eed40cb039a service cinder] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:35.665413 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1bc8fc-54ea-41c1-8e4b-966f1e762b6e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c4def55ac0e48b29836bc5c64bc7402', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:35.666745 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1bc8fc-54ea-41c1-8e4b-966f1e762b6e service cinder] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:35.666745 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1bc8fc-54ea-41c1-8e4b-966f1e762b6e service cinder] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:35.666745 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1bc8fc-54ea-41c1-8e4b-966f1e762b6e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:35.676252 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1bc8fc-54ea-41c1-8e4b-966f1e762b6e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:35.677700 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1bc8fc-54ea-41c1-8e4b-966f1e762b6e service cinder] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:35.688143 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1165/2328] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:43:35 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:43:45.683563 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=6 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:44:40.933247 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b38de62-dc86-49e1-92d3-81743cde5395 service neutron] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:44:40.943139 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b38de62-dc86-49e1-92d3-81743cde5395 service neutron] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:44:40.946714 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10264066-55ab-448b-bedd-458f54d76622 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c4def55ac0e48b29836bc5c64bc7402', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:44:40.947309 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10264066-55ab-448b-bedd-458f54d76622 service cinder] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:40.947589 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10264066-55ab-448b-bedd-458f54d76622 service cinder] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:40.947874 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10264066-55ab-448b-bedd-458f54d76622 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:41.006677 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10264066-55ab-448b-bedd-458f54d76622 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:44:41.008340 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10264066-55ab-448b-bedd-458f54d76622 service cinder] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:44:41.017516 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1164/2329] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:44:40 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:44:41.473697 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1bc8fc-54ea-41c1-8e4b-966f1e762b6e service cinder] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:44:41.482579 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1bc8fc-54ea-41c1-8e4b-966f1e762b6e service cinder] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:44:41.486391 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4461e506-2d59-4901-b107-7d02fccd506d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c4def55ac0e48b29836bc5c64bc7402', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:44:41.486988 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4461e506-2d59-4901-b107-7d02fccd506d service cinder] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:41.487296 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4461e506-2d59-4901-b107-7d02fccd506d service cinder] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:41.487569 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4461e506-2d59-4901-b107-7d02fccd506d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:41.489313 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-4461e506-2d59-4901-b107-7d02fccd506d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:44:41.549980 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4461e506-2d59-4901-b107-7d02fccd506d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:44:41.551484 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4461e506-2d59-4901-b107-7d02fccd506d service cinder] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:44:41.561249 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1166/2330] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:44:41 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:44:41.643014 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10264066-55ab-448b-bedd-458f54d76622 service cinder] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:44:41.651837 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10264066-55ab-448b-bedd-458f54d76622 service cinder] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:44:41.655517 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2eeaaf-83ab-42d5-85ad-c1e5f1318793 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c4def55ac0e48b29836bc5c64bc7402', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:44:41.656135 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2eeaaf-83ab-42d5-85ad-c1e5f1318793 service cinder] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:41.656521 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2eeaaf-83ab-42d5-85ad-c1e5f1318793 service cinder] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:41.656800 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2eeaaf-83ab-42d5-85ad-c1e5f1318793 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:41.658514 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-2f2eeaaf-83ab-42d5-85ad-c1e5f1318793 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:44:41.722707 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2eeaaf-83ab-42d5-85ad-c1e5f1318793 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:44:41.724756 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2eeaaf-83ab-42d5-85ad-c1e5f1318793 service cinder] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:44:41.733733 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1165/2331] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:44:41 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:44:51.556918 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=28 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:44:51.729875 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=56 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:45:56.952227 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4461e506-2d59-4901-b107-7d02fccd506d service cinder] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:45:56.953768 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-4461e506-2d59-4901-b107-7d02fccd506d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:45:57.106534 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4461e506-2d59-4901-b107-7d02fccd506d service cinder] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:45:57.111804 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca0462b-6514-44a4-aaf6-e30365ae5eed service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abe9d8c9291845dfa4ef55034e5c9e32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:45:57.112410 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca0462b-6514-44a4-aaf6-e30365ae5eed service placement] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:45:57.112709 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca0462b-6514-44a4-aaf6-e30365ae5eed service placement] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:45:57.112977 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca0462b-6514-44a4-aaf6-e30365ae5eed service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:45:57.114709 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-fca0462b-6514-44a4-aaf6-e30365ae5eed service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:45:57.126937 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca0462b-6514-44a4-aaf6-e30365ae5eed service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:45:57.128562 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca0462b-6514-44a4-aaf6-e30365ae5eed service placement] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:45:57.139837 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1167/2332] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:45:56 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:46:07.134887 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=58 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:46:43.465216 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2eeaaf-83ab-42d5-85ad-c1e5f1318793 service cinder] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:46:43.466773 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-2f2eeaaf-83ab-42d5-85ad-c1e5f1318793 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:46:43.499370 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca0462b-6514-44a4-aaf6-e30365ae5eed service placement] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:46:43.510466 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2eeaaf-83ab-42d5-85ad-c1e5f1318793 service cinder] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:46:43.515293 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c41425-0629-4f0e-a390-69714a1bae44 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bb6c79db75e4d169331c02133d1ec32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:46:43.516196 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c41425-0629-4f0e-a390-69714a1bae44 service neutron] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:46:43.516630 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c41425-0629-4f0e-a390-69714a1bae44 service neutron] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:46:43.517023 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c41425-0629-4f0e-a390-69714a1bae44 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:46:43.518801 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-c5c41425-0629-4f0e-a390-69714a1bae44 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:46:43.520982 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca0462b-6514-44a4-aaf6-e30365ae5eed service placement] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:46:43.524783 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d702c972-61b5-404b-aa26-db048f82fd1a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bb6c79db75e4d169331c02133d1ec32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:46:43.525379 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d702c972-61b5-404b-aa26-db048f82fd1a service neutron] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:46:43.525653 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d702c972-61b5-404b-aa26-db048f82fd1a service neutron] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:46:43.525925 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d702c972-61b5-404b-aa26-db048f82fd1a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:46:43.527690 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-d702c972-61b5-404b-aa26-db048f82fd1a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:46:43.613999 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c41425-0629-4f0e-a390-69714a1bae44 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:46:43.615265 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d702c972-61b5-404b-aa26-db048f82fd1a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:46:43.615504 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c41425-0629-4f0e-a390-69714a1bae44 service neutron] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:46:43.616706 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d702c972-61b5-404b-aa26-db048f82fd1a service neutron] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:46:43.637229 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1166/2333] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:46:43 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:46:43.637663 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1168/2334] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:46:43 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 06:46:44.623374 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c41425-0629-4f0e-a390-69714a1bae44 service neutron] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:46:44.625409 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-c5c41425-0629-4f0e-a390-69714a1bae44 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:46:44.645457 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c41425-0629-4f0e-a390-69714a1bae44 service neutron] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:46:44.652803 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8667d9ef-9417-4ab9-b7d7-52e791b238b3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb6d3743340241f98cacb9e0d063298c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:46:44.653426 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8667d9ef-9417-4ab9-b7d7-52e791b238b3 service nova] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:46:44.653708 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8667d9ef-9417-4ab9-b7d7-52e791b238b3 service nova] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:46:44.653979 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8667d9ef-9417-4ab9-b7d7-52e791b238b3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:46:44.655693 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-8667d9ef-9417-4ab9-b7d7-52e791b238b3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:46:44.668326 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8667d9ef-9417-4ab9-b7d7-52e791b238b3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:46:44.669966 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8667d9ef-9417-4ab9-b7d7-52e791b238b3 service nova] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:46:44.680879 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1167/2335] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:46:44 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:46:45.406245 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d702c972-61b5-404b-aa26-db048f82fd1a service neutron] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:46:45.417431 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d702c972-61b5-404b-aa26-db048f82fd1a service neutron] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:46:45.423141 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd59a4b-8c0d-4746-aa66-f25d5dfbafa7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb6d3743340241f98cacb9e0d063298c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:46:45.423770 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd59a4b-8c0d-4746-aa66-f25d5dfbafa7 service nova] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:46:45.424047 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd59a4b-8c0d-4746-aa66-f25d5dfbafa7 service nova] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:46:45.424318 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd59a4b-8c0d-4746-aa66-f25d5dfbafa7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:46:45.426348 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-2fd59a4b-8c0d-4746-aa66-f25d5dfbafa7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:46:45.440022 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd59a4b-8c0d-4746-aa66-f25d5dfbafa7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:46:45.441634 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd59a4b-8c0d-4746-aa66-f25d5dfbafa7 service nova] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:46:45.452923 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1169/2336] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:46:45 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:46:48.678794 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8667d9ef-9417-4ab9-b7d7-52e791b238b3 service nova] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:46:48.692732 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8667d9ef-9417-4ab9-b7d7-52e791b238b3 service nova] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:46:48.696445 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ccb4f2-67a0-4ff3-be36-865cc6b3e723 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bb6c79db75e4d169331c02133d1ec32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:46:48.697260 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ccb4f2-67a0-4ff3-be36-865cc6b3e723 service neutron] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:46:48.697411 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ccb4f2-67a0-4ff3-be36-865cc6b3e723 service neutron] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:46:48.697671 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ccb4f2-67a0-4ff3-be36-865cc6b3e723 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:46:48.700187 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-e6ccb4f2-67a0-4ff3-be36-865cc6b3e723 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:46:48.782978 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ccb4f2-67a0-4ff3-be36-865cc6b3e723 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:46:48.785438 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ccb4f2-67a0-4ff3-be36-865cc6b3e723 service neutron] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:46:48.795187 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1168/2337] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:46:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:46:51.571792 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1c5898-e6c7-4b74-9fed-4d1415ac1951 None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:46:51.571792 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1c5898-e6c7-4b74-9fed-4d1415ac1951 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:46:51.571792 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1c5898-e6c7-4b74-9fed-4d1415ac1951 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:46:51.591311 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-fb1c5898-e6c7-4b74-9fed-4d1415ac1951 None None] MFA Rules not processed for user `6abfd0a28f9e482badae7018f1b7f9a1`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:46:51.592393 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-fb1c5898-e6c7-4b74-9fed-4d1415ac1951 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:46:51.599735 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1170/2338] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:46:51 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:46:51.976473 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ccb4f2-67a0-4ff3-be36-865cc6b3e723 service neutron] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:46:51.985612 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ccb4f2-67a0-4ff3-be36-865cc6b3e723 service neutron] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:46:51.989376 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e114124b-dcda-40c9-addb-e8fc8be185f9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bb6c79db75e4d169331c02133d1ec32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:46:51.990209 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e114124b-dcda-40c9-addb-e8fc8be185f9 service neutron] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:46:51.990551 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e114124b-dcda-40c9-addb-e8fc8be185f9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:46:51.990936 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e114124b-dcda-40c9-addb-e8fc8be185f9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:46:51.992911 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-e114124b-dcda-40c9-addb-e8fc8be185f9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:46:52.003437 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e114124b-dcda-40c9-addb-e8fc8be185f9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:46:52.004993 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e114124b-dcda-40c9-addb-e8fc8be185f9 service neutron] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:46:52.008200 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:46:52.014544 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1169/2339] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:46:51 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:46:52.463100 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1c5898-e6c7-4b74-9fed-4d1415ac1951 None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:46:52.485347 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1c5898-e6c7-4b74-9fed-4d1415ac1951 None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:46:52.490701 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db6d016-da68-4adf-b4f6-7d3851532209 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abe9d8c9291845dfa4ef55034e5c9e32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:46:52.491316 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db6d016-da68-4adf-b4f6-7d3851532209 service placement] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:46:52.491598 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db6d016-da68-4adf-b4f6-7d3851532209 service placement] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:46:52.491847 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db6d016-da68-4adf-b4f6-7d3851532209 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:46:52.494466 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-3db6d016-da68-4adf-b4f6-7d3851532209 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:46:52.509697 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db6d016-da68-4adf-b4f6-7d3851532209 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:46:52.511532 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db6d016-da68-4adf-b4f6-7d3851532209 service placement] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:46:52.522741 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1171/2340] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:46:52 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:46:52.990195 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e114124b-dcda-40c9-addb-e8fc8be185f9 service neutron] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:46:52.994994 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-e114124b-dcda-40c9-addb-e8fc8be185f9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:46:53.007831 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e114124b-dcda-40c9-addb-e8fc8be185f9 service neutron] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:46:53.013237 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e64d241-f24f-4268-a0ed-5cd2cc81b4b7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abe9d8c9291845dfa4ef55034e5c9e32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:46:53.014037 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e64d241-f24f-4268-a0ed-5cd2cc81b4b7 service placement] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:46:53.014394 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e64d241-f24f-4268-a0ed-5cd2cc81b4b7 service placement] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:46:53.014738 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e64d241-f24f-4268-a0ed-5cd2cc81b4b7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:46:53.016670 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-8e64d241-f24f-4268-a0ed-5cd2cc81b4b7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:46:53.029189 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e64d241-f24f-4268-a0ed-5cd2cc81b4b7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:46:53.030813 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e64d241-f24f-4268-a0ed-5cd2cc81b4b7 service placement] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:46:53.040687 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1170/2341] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:46:52 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:46:53.187957 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db6d016-da68-4adf-b4f6-7d3851532209 service placement] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:46:53.189462 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-3db6d016-da68-4adf-b4f6-7d3851532209 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:46:53.204189 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db6d016-da68-4adf-b4f6-7d3851532209 service placement] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:46:53.208644 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab7b3b1-2be5-498b-ad22-5643d61283f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:46:53.209256 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab7b3b1-2be5-498b-ad22-5643d61283f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:46:53.209531 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab7b3b1-2be5-498b-ad22-5643d61283f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:46:53.209821 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab7b3b1-2be5-498b-ad22-5643d61283f6 admin admin] PATH_INFO: `/v3/users/069dc6f771a74dd4ab22d9d5e68e3cc9` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:46:53.210919 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab7b3b1-2be5-498b-ad22-5643d61283f6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=069dc6f771a74dd4ab22d9d5e68e3cc9)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:46:53.212849 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab7b3b1-2be5-498b-ad22-5643d61283f6 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:46:53.399861 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-eab7b3b1-2be5-498b-ad22-5643d61283f6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '069dc6f771a74dd4ab22d9d5e68e3cc9'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:46:53.407326 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-eab7b3b1-2be5-498b-ad22-5643d61283f6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '069dc6f771a74dd4ab22d9d5e68e3cc9'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:46:53.408013 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-eab7b3b1-2be5-498b-ad22-5643d61283f6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '069dc6f771a74dd4ab22d9d5e68e3cc9'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:46:53.408646 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-eab7b3b1-2be5-498b-ad22-5643d61283f6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '069dc6f771a74dd4ab22d9d5e68e3cc9'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:46:53.523406 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-eab7b3b1-2be5-498b-ad22-5643d61283f6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '069dc6f771a74dd4ab22d9d5e68e3cc9'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:46:53.537056 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:46:53.549341 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1172/2342] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Aug 24 06:46:53 2022] DELETE /identity/v3/users/069dc6f771a74dd4ab22d9d5e68e3cc9 => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:46:53.844871 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e64d241-f24f-4268-a0ed-5cd2cc81b4b7 service placement] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:46:53.858401 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e64d241-f24f-4268-a0ed-5cd2cc81b4b7 service placement] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:46:53.919179 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20eded3-5361-4f3c-b27c-0337db050615 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:46:53.919816 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20eded3-5361-4f3c-b27c-0337db050615 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:46:53.920094 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20eded3-5361-4f3c-b27c-0337db050615 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:46:53.920363 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20eded3-5361-4f3c-b27c-0337db050615 admin admin] PATH_INFO: `/v3/projects/c467d980c962456c9fa0e28364efd2a3` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:46:53.921526 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20eded3-5361-4f3c-b27c-0337db050615 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c467d980c962456c9fa0e28364efd2a3)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:46:53.923469 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20eded3-5361-4f3c-b27c-0337db050615 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:46:54.044483 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-a20eded3-5361-4f3c-b27c-0337db050615 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c467d980c962456c9fa0e28364efd2a3', 'request_id': 'req-a20eded3-5361-4f3c-b27c-0337db050615'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:46:54.050632 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-a20eded3-5361-4f3c-b27c-0337db050615 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c467d980c962456c9fa0e28364efd2a3', 'request_id': 'req-a20eded3-5361-4f3c-b27c-0337db050615'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:46:54.057141 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-a20eded3-5361-4f3c-b27c-0337db050615 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c467d980c962456c9fa0e28364efd2a3', 'request_id': 'req-a20eded3-5361-4f3c-b27c-0337db050615'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:46:54.158456 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-a20eded3-5361-4f3c-b27c-0337db050615 admin admin] The token cache is being invalidate because project c467d980c962456c9fa0e28364efd2a3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81852) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:46:54.158683 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-a20eded3-5361-4f3c-b27c-0337db050615 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:46:54.159311 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-a20eded3-5361-4f3c-b27c-0337db050615 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:46:54.160925 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1171/2343] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Aug 24 06:46:53 2022] DELETE /identity/v3/projects/c467d980c962456c9fa0e28364efd2a3 => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:46:54.166876 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a563cf-5a53-4082-9f92-614e3256f0e0 None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:46:54.167207 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a563cf-5a53-4082-9f92-614e3256f0e0 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:46:54.167463 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a563cf-5a53-4082-9f92-614e3256f0e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:46:54.187570 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-b4a563cf-5a53-4082-9f92-614e3256f0e0 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:46:54.189167 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-b4a563cf-5a53-4082-9f92-614e3256f0e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:46:54.253964 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1173/2344] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:46:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:46:54.316861 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552636d3-7581-4119-b03b-44b83cf61c51 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:46:54.317140 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552636d3-7581-4119-b03b-44b83cf61c51 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:46:54.317415 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552636d3-7581-4119-b03b-44b83cf61c51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:46:54.337646 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-552636d3-7581-4119-b03b-44b83cf61c51 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:46:54.338928 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-552636d3-7581-4119-b03b-44b83cf61c51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:46:54.349160 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1172/2345] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:46:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:46:54.351230 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a563cf-5a53-4082-9f92-614e3256f0e0 None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:46:54.361003 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a563cf-5a53-4082-9f92-614e3256f0e0 None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:46:54.365684 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87d13b6-4f1a-471e-be26-df35160e7312 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:46:54.366278 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87d13b6-4f1a-471e-be26-df35160e7312 admin admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:46:54.366558 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87d13b6-4f1a-471e-be26-df35160e7312 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:46:54.366830 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87d13b6-4f1a-471e-be26-df35160e7312 admin admin] PATH_INFO: `/v3/domains` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:46:54.367470 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87d13b6-4f1a-471e-be26-df35160e7312 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:46:54.367881 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87d13b6-4f1a-471e-be26-df35160e7312 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:46:54.369922 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87d13b6-4f1a-471e-be26-df35160e7312 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:46:54.381513 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1174/2346] 10.222.0.62 () {58 vars in 1153 bytes} [Wed Aug 24 06:46:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:46:54.386436 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552636d3-7581-4119-b03b-44b83cf61c51 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:46:54.396473 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552636d3-7581-4119-b03b-44b83cf61c51 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:46:54.401187 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da157ed7-6e44-4238-97aa-f774eacc6131 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:46:54.401787 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da157ed7-6e44-4238-97aa-f774eacc6131 admin admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:46:54.402139 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da157ed7-6e44-4238-97aa-f774eacc6131 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:46:54.402411 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da157ed7-6e44-4238-97aa-f774eacc6131 admin admin] PATH_INFO: `/v3/projects` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:46:54.403373 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da157ed7-6e44-4238-97aa-f774eacc6131 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:46:54.405009 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da157ed7-6e44-4238-97aa-f774eacc6131 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:46:54.490981 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1173/2347] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Aug 24 06:46:54 2022] POST /identity/v3/projects => generated 369 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 06:46:54.495830 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87d13b6-4f1a-471e-be26-df35160e7312 admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:46:54.508647 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87d13b6-4f1a-471e-be26-df35160e7312 admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:46:54.565813 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44c8bc6-1551-4e89-9610-e612e38b07d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:46:54.566416 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44c8bc6-1551-4e89-9610-e612e38b07d7 admin admin] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:46:54.566692 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44c8bc6-1551-4e89-9610-e612e38b07d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:46:54.566964 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44c8bc6-1551-4e89-9610-e612e38b07d7 admin admin] PATH_INFO: `/v3/users` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:46:54.567602 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44c8bc6-1551-4e89-9610-e612e38b07d7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:46:54.569551 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44c8bc6-1551-4e89-9610-e612e38b07d7 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:46:54.691165 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1175/2348] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Aug 24 06:46:54 2022] POST /identity/v3/users => generated 285 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:46:54.695726 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da157ed7-6e44-4238-97aa-f774eacc6131 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:46:54.706569 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da157ed7-6e44-4238-97aa-f774eacc6131 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:46:54.712030 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f9b31e-37bf-4c69-b53e-ed6da4b2eaad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:46:54.713656 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f9b31e-37bf-4c69-b53e-ed6da4b2eaad admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:46:54.714016 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f9b31e-37bf-4c69-b53e-ed6da4b2eaad admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:46:54.714296 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f9b31e-37bf-4c69-b53e-ed6da4b2eaad admin admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:46:54.714963 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f9b31e-37bf-4c69-b53e-ed6da4b2eaad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:46:54.716653 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f9b31e-37bf-4c69-b53e-ed6da4b2eaad admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:46:54.730857 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1174/2349] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Aug 24 06:46:54 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:46:54.734567 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44c8bc6-1551-4e89-9610-e612e38b07d7 admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:46:54.745879 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44c8bc6-1551-4e89-9610-e612e38b07d7 admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:46:54.750521 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424f5735-b29b-47c7-9694-d6808cb54155 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:46:54.751145 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424f5735-b29b-47c7-9694-d6808cb54155 admin admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:46:54.751425 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424f5735-b29b-47c7-9694-d6808cb54155 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:46:54.751699 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424f5735-b29b-47c7-9694-d6808cb54155 admin admin] PATH_INFO: `/v3/roles` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:46:54.752412 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424f5735-b29b-47c7-9694-d6808cb54155 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:46:54.759276 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424f5735-b29b-47c7-9694-d6808cb54155 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:46:54.767550 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1176/2350] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Aug 24 06:46:54 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:46:54.771763 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f9b31e-37bf-4c69-b53e-ed6da4b2eaad admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:46:54.782597 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f9b31e-37bf-4c69-b53e-ed6da4b2eaad admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:46:54.788186 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc74cbc-cc31-42ed-8bc3-787bfbbc5fc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:46:54.789104 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc74cbc-cc31-42ed-8bc3-787bfbbc5fc3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:46:54.789499 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc74cbc-cc31-42ed-8bc3-787bfbbc5fc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:46:54.789871 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc74cbc-cc31-42ed-8bc3-787bfbbc5fc3 admin admin] PATH_INFO: `/v3/projects/d3bbe217a67e4d778f1a48317c8301b7/users/f31dd95cc896458a9187886e5b66ca46/roles/580aa7dbb38d4cff8f9d1bb8983f99ef` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:46:54.806238 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc74cbc-cc31-42ed-8bc3-787bfbbc5fc3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d3bbe217a67e4d778f1a48317c8301b7, user_id=f31dd95cc896458a9187886e5b66ca46, role_id=580aa7dbb38d4cff8f9d1bb8983f99ef)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:46:54.809567 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc74cbc-cc31-42ed-8bc3-787bfbbc5fc3 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:46:54.914171 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1175/2351] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Aug 24 06:46:54 2022] PUT /identity/v3/projects/d3bbe217a67e4d778f1a48317c8301b7/users/f31dd95cc896458a9187886e5b66ca46/roles/580aa7dbb38d4cff8f9d1bb8983f99ef => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 06:46:54.924332 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d34b06-a3a0-453f-b023-7e52581978cc None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:46:54.925385 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d34b06-a3a0-453f-b023-7e52581978cc None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:46:54.925385 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d34b06-a3a0-453f-b023-7e52581978cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:46:54.960788 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-94d34b06-a3a0-453f-b023-7e52581978cc None None] MFA Rules not processed for user `f31dd95cc896458a9187886e5b66ca46`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:46:54.962005 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-94d34b06-a3a0-453f-b023-7e52581978cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:46:55.011198 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:46:55.032448 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1177/2352] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:46:54 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:46:55.044023 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc74cbc-cc31-42ed-8bc3-787bfbbc5fc3 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:46:55.045923 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-bcc74cbc-cc31-42ed-8bc3-787bfbbc5fc3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:46:55.067857 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:46:55.130703 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc74cbc-cc31-42ed-8bc3-787bfbbc5fc3 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:46:55.136920 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308e35e0-8bdf-493f-98d8-df785e96a816 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb6d3743340241f98cacb9e0d063298c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:46:55.137526 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308e35e0-8bdf-493f-98d8-df785e96a816 service nova] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:46:55.137804 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308e35e0-8bdf-493f-98d8-df785e96a816 service nova] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:46:55.138075 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308e35e0-8bdf-493f-98d8-df785e96a816 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:46:55.146565 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308e35e0-8bdf-493f-98d8-df785e96a816 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:46:55.148148 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308e35e0-8bdf-493f-98d8-df785e96a816 service nova] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:46:55.157242 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1176/2353] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:46:55 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:47:02.352222 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d34b06-a3a0-453f-b023-7e52581978cc None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:47:02.353781 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-94d34b06-a3a0-453f-b023-7e52581978cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:47:02.369505 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d34b06-a3a0-453f-b023-7e52581978cc None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:47:02.374956 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec433658-a64e-4b59-ab23-33203963e8c1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb6d3743340241f98cacb9e0d063298c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:47:02.375534 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec433658-a64e-4b59-ab23-33203963e8c1 service nova] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:47:02.375815 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec433658-a64e-4b59-ab23-33203963e8c1 service nova] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:47:02.376088 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec433658-a64e-4b59-ab23-33203963e8c1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:47:02.377898 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-ec433658-a64e-4b59-ab23-33203963e8c1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:47:02.390149 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec433658-a64e-4b59-ab23-33203963e8c1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:47:02.391788 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec433658-a64e-4b59-ab23-33203963e8c1 service nova] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:47:02.402749 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1178/2354] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:47:02 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:47:05.655716 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=29 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:47:12.397773 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=11 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:47:39.595189 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308e35e0-8bdf-493f-98d8-df785e96a816 service nova] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:47:39.607832 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308e35e0-8bdf-493f-98d8-df785e96a816 service nova] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:47:39.613573 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf612fd-e553-4b68-80c0-1e6fa49bdd3b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb6d3743340241f98cacb9e0d063298c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:47:39.614171 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf612fd-e553-4b68-80c0-1e6fa49bdd3b service nova] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:47:39.614453 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf612fd-e553-4b68-80c0-1e6fa49bdd3b service nova] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:47:39.614728 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf612fd-e553-4b68-80c0-1e6fa49bdd3b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:47:39.616466 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-faf612fd-e553-4b68-80c0-1e6fa49bdd3b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:47:39.696474 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf612fd-e553-4b68-80c0-1e6fa49bdd3b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:47:39.698122 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf612fd-e553-4b68-80c0-1e6fa49bdd3b service nova] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:47:39.723712 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1177/2355] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:47:39 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:47:48.726681 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec433658-a64e-4b59-ab23-33203963e8c1 service nova] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:47:48.728962 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-ec433658-a64e-4b59-ab23-33203963e8c1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:47:48.800087 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec433658-a64e-4b59-ab23-33203963e8c1 service nova] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:47:48.803591 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598af291-d4c8-4c56-97f2-ee0f5bfa46ea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bb6c79db75e4d169331c02133d1ec32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:47:48.807814 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598af291-d4c8-4c56-97f2-ee0f5bfa46ea service neutron] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:47:48.807814 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598af291-d4c8-4c56-97f2-ee0f5bfa46ea service neutron] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:47:48.807814 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598af291-d4c8-4c56-97f2-ee0f5bfa46ea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:47:48.807814 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-598af291-d4c8-4c56-97f2-ee0f5bfa46ea service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:47:48.816427 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598af291-d4c8-4c56-97f2-ee0f5bfa46ea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:47:48.817881 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598af291-d4c8-4c56-97f2-ee0f5bfa46ea service neutron] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:47:48.825713 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1179/2356] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:47:48 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:47:49.720828 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=36 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:47:55.095269 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf612fd-e553-4b68-80c0-1e6fa49bdd3b service nova] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:47:55.156039 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf612fd-e553-4b68-80c0-1e6fa49bdd3b service nova] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:47:55.160793 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1cce82-fabc-4621-b95c-444d04340b1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:47:55.161416 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1cce82-fabc-4621-b95c-444d04340b1d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:47:55.161696 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1cce82-fabc-4621-b95c-444d04340b1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:47:55.161968 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1cce82-fabc-4621-b95c-444d04340b1d admin admin] PATH_INFO: `/v3/users/f31dd95cc896458a9187886e5b66ca46` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:47:55.163083 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1cce82-fabc-4621-b95c-444d04340b1d admin admin] RBAC: Authorizing `identity:delete_user(user_id=f31dd95cc896458a9187886e5b66ca46)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:47:55.165032 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1cce82-fabc-4621-b95c-444d04340b1d admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:47:55.393945 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-5d1cce82-fabc-4621-b95c-444d04340b1d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f31dd95cc896458a9187886e5b66ca46'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:47:55.400777 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-5d1cce82-fabc-4621-b95c-444d04340b1d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f31dd95cc896458a9187886e5b66ca46'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:47:55.401443 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-5d1cce82-fabc-4621-b95c-444d04340b1d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f31dd95cc896458a9187886e5b66ca46'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:47:55.531982 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-5d1cce82-fabc-4621-b95c-444d04340b1d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f31dd95cc896458a9187886e5b66ca46'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:47:55.544732 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-5d1cce82-fabc-4621-b95c-444d04340b1d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f31dd95cc896458a9187886e5b66ca46'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:47:55.550757 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1178/2357] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Aug 24 06:47:55 2022] DELETE /identity/v3/users/f31dd95cc896458a9187886e5b66ca46 => generated 0 bytes in 456 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:47:55.555455 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2f33eb-d451-4bf5-914d-292bef7e0592 None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:47:55.555963 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2f33eb-d451-4bf5-914d-292bef7e0592 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:47:55.556080 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2f33eb-d451-4bf5-914d-292bef7e0592 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:47:55.575346 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-cb2f33eb-d451-4bf5-914d-292bef7e0592 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:47:55.576558 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-cb2f33eb-d451-4bf5-914d-292bef7e0592 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:47:55.635518 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1180/2358] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:47:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:47:55.645942 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1cce82-fabc-4621-b95c-444d04340b1d admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:47:55.647401 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-5d1cce82-fabc-4621-b95c-444d04340b1d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:47:55.709901 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1cce82-fabc-4621-b95c-444d04340b1d admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:47:55.713625 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44fba21-0d8a-4752-92da-8e2c83e9bc24 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bb6c79db75e4d169331c02133d1ec32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:47:55.714236 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44fba21-0d8a-4752-92da-8e2c83e9bc24 service neutron] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:47:55.714514 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44fba21-0d8a-4752-92da-8e2c83e9bc24 service neutron] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:47:55.714788 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44fba21-0d8a-4752-92da-8e2c83e9bc24 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:47:55.724208 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44fba21-0d8a-4752-92da-8e2c83e9bc24 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:47:55.725684 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44fba21-0d8a-4752-92da-8e2c83e9bc24 service neutron] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:47:55.735421 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1179/2359] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:47:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:47:56.169511 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2f33eb-d451-4bf5-914d-292bef7e0592 None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:47:56.172340 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-cb2f33eb-d451-4bf5-914d-292bef7e0592 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:47:56.183283 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2f33eb-d451-4bf5-914d-292bef7e0592 None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:47:56.189002 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9456443-4899-46f0-848e-ebf446a7db99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:47:56.190751 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9456443-4899-46f0-848e-ebf446a7db99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:47:56.191193 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9456443-4899-46f0-848e-ebf446a7db99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:47:56.191471 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9456443-4899-46f0-848e-ebf446a7db99 admin admin] PATH_INFO: `/v3/projects/d3bbe217a67e4d778f1a48317c8301b7` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:47:56.192696 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9456443-4899-46f0-848e-ebf446a7db99 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d3bbe217a67e4d778f1a48317c8301b7)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:47:56.195579 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9456443-4899-46f0-848e-ebf446a7db99 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:47:56.408121 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-b9456443-4899-46f0-848e-ebf446a7db99 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd3bbe217a67e4d778f1a48317c8301b7', 'request_id': 'req-b9456443-4899-46f0-848e-ebf446a7db99'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:47:56.413488 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-b9456443-4899-46f0-848e-ebf446a7db99 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd3bbe217a67e4d778f1a48317c8301b7', 'request_id': 'req-b9456443-4899-46f0-848e-ebf446a7db99'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:47:56.419473 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-b9456443-4899-46f0-848e-ebf446a7db99 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd3bbe217a67e4d778f1a48317c8301b7', 'request_id': 'req-b9456443-4899-46f0-848e-ebf446a7db99'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:47:56.502983 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-b9456443-4899-46f0-848e-ebf446a7db99 admin admin] The token cache is being invalidate because project d3bbe217a67e4d778f1a48317c8301b7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81851) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:47:56.503252 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-b9456443-4899-46f0-848e-ebf446a7db99 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:47:56.503859 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-b9456443-4899-46f0-848e-ebf446a7db99 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:47:56.505373 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1181/2360] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Aug 24 06:47:56 2022] DELETE /identity/v3/projects/d3bbe217a67e4d778f1a48317c8301b7 => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:47:56.510293 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1726f5b-4dae-4b65-9cc1-7c2d29e30191 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:47:56.510776 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1726f5b-4dae-4b65-9cc1-7c2d29e30191 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:47:56.510879 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1726f5b-4dae-4b65-9cc1-7c2d29e30191 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:47:56.529559 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-c1726f5b-4dae-4b65-9cc1-7c2d29e30191 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:47:56.530734 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-c1726f5b-4dae-4b65-9cc1-7c2d29e30191 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:47:56.548053 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:47:56.589820 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1180/2361] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:47:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:47:56.598210 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d22ef1b-4f1b-44f6-b8f8-08a30855501a None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:47:56.598796 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d22ef1b-4f1b-44f6-b8f8-08a30855501a None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:47:56.598796 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d22ef1b-4f1b-44f6-b8f8-08a30855501a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:47:56.618513 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-8d22ef1b-4f1b-44f6-b8f8-08a30855501a None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:47:56.619702 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-8d22ef1b-4f1b-44f6-b8f8-08a30855501a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:47:56.627479 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1182/2362] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:47:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:47:56.631925 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1726f5b-4dae-4b65-9cc1-7c2d29e30191 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:47:56.642227 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1726f5b-4dae-4b65-9cc1-7c2d29e30191 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:47:56.647038 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323bd29a-c99c-4091-8877-221e66579c94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:47:56.647649 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323bd29a-c99c-4091-8877-221e66579c94 admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:47:56.647927 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323bd29a-c99c-4091-8877-221e66579c94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:47:56.648197 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323bd29a-c99c-4091-8877-221e66579c94 admin admin] PATH_INFO: `/v3/domains` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:47:56.648882 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323bd29a-c99c-4091-8877-221e66579c94 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:47:56.649185 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323bd29a-c99c-4091-8877-221e66579c94 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:47:56.650808 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323bd29a-c99c-4091-8877-221e66579c94 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:47:56.663188 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1181/2363] 10.222.0.62 () {58 vars in 1153 bytes} [Wed Aug 24 06:47:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:47:56.668046 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d22ef1b-4f1b-44f6-b8f8-08a30855501a None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:47:56.678618 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d22ef1b-4f1b-44f6-b8f8-08a30855501a None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:47:56.683362 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202922e5-15f8-4827-bb78-0d53083062e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:47:56.683961 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202922e5-15f8-4827-bb78-0d53083062e9 admin admin] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:47:56.684279 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202922e5-15f8-4827-bb78-0d53083062e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:47:56.684590 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202922e5-15f8-4827-bb78-0d53083062e9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:47:56.685496 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202922e5-15f8-4827-bb78-0d53083062e9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:47:56.687426 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202922e5-15f8-4827-bb78-0d53083062e9 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:47:56.693096 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:47:56.815778 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1183/2364] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Aug 24 06:47:56 2022] POST /identity/v3/projects => generated 371 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:47:56.820000 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323bd29a-c99c-4091-8877-221e66579c94 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:47:56.830329 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323bd29a-c99c-4091-8877-221e66579c94 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:47:56.885072 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc996936-3cd7-4022-ad14-a096455a4ace admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:47:56.885685 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc996936-3cd7-4022-ad14-a096455a4ace admin admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:47:56.885967 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc996936-3cd7-4022-ad14-a096455a4ace admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:47:56.886253 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc996936-3cd7-4022-ad14-a096455a4ace admin admin] PATH_INFO: `/v3/users` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:47:56.886969 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc996936-3cd7-4022-ad14-a096455a4ace admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:47:56.888957 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc996936-3cd7-4022-ad14-a096455a4ace admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:47:56.997967 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1182/2365] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Aug 24 06:47:56 2022] POST /identity/v3/users => generated 286 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:47:57.000444 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202922e5-15f8-4827-bb78-0d53083062e9 admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:47:57.010929 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202922e5-15f8-4827-bb78-0d53083062e9 admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:47:57.015628 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ba35f3-b897-43bb-bf17-810ef6b48f1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:47:57.016757 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ba35f3-b897-43bb-bf17-810ef6b48f1e admin admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:47:57.017034 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ba35f3-b897-43bb-bf17-810ef6b48f1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:47:57.017302 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ba35f3-b897-43bb-bf17-810ef6b48f1e admin admin] PATH_INFO: `/v3/roles` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:47:57.017939 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ba35f3-b897-43bb-bf17-810ef6b48f1e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:47:57.019522 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ba35f3-b897-43bb-bf17-810ef6b48f1e admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:47:57.029230 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1184/2366] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Aug 24 06:47:56 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:47:57.033717 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc996936-3cd7-4022-ad14-a096455a4ace admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:47:57.044094 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc996936-3cd7-4022-ad14-a096455a4ace admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:47:57.048988 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3033f809-5ecc-46e2-bca3-d87791cbbca6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:47:57.049583 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3033f809-5ecc-46e2-bca3-d87791cbbca6 admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:47:57.049864 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3033f809-5ecc-46e2-bca3-d87791cbbca6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:47:57.050134 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3033f809-5ecc-46e2-bca3-d87791cbbca6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:47:57.050770 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3033f809-5ecc-46e2-bca3-d87791cbbca6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:47:57.052357 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3033f809-5ecc-46e2-bca3-d87791cbbca6 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:47:57.063780 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1183/2367] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Aug 24 06:47:57 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:47:57.068174 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ba35f3-b897-43bb-bf17-810ef6b48f1e admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:47:57.078088 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ba35f3-b897-43bb-bf17-810ef6b48f1e admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:47:57.083302 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8d503a-d056-4e6e-97ed-6378a1b63bc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:47:57.083943 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8d503a-d056-4e6e-97ed-6378a1b63bc0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:47:57.084220 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8d503a-d056-4e6e-97ed-6378a1b63bc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:47:57.084527 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8d503a-d056-4e6e-97ed-6378a1b63bc0 admin admin] PATH_INFO: `/v3/projects/b3d00da71a4447d38459b39369bde7bc/users/6aeae9bc6673480daa9c74c915b11a1d/roles/580aa7dbb38d4cff8f9d1bb8983f99ef` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:47:57.098431 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8d503a-d056-4e6e-97ed-6378a1b63bc0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b3d00da71a4447d38459b39369bde7bc, user_id=6aeae9bc6673480daa9c74c915b11a1d, role_id=580aa7dbb38d4cff8f9d1bb8983f99ef)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:47:57.101096 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8d503a-d056-4e6e-97ed-6378a1b63bc0 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:47:57.219460 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1185/2368] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Aug 24 06:47:57 2022] PUT /identity/v3/projects/b3d00da71a4447d38459b39369bde7bc/users/6aeae9bc6673480daa9c74c915b11a1d/roles/580aa7dbb38d4cff8f9d1bb8983f99ef => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:47:57.230771 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9029523-4b81-4a7f-bcc4-4838b26ff14e None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:47:57.231055 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9029523-4b81-4a7f-bcc4-4838b26ff14e None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:47:57.231330 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9029523-4b81-4a7f-bcc4-4838b26ff14e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:47:57.260533 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-a9029523-4b81-4a7f-bcc4-4838b26ff14e None None] MFA Rules not processed for user `6aeae9bc6673480daa9c74c915b11a1d`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:47:57.261710 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-a9029523-4b81-4a7f-bcc4-4838b26ff14e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:47:57.333917 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1184/2369] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:47:57 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:47:57.338750 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8d503a-d056-4e6e-97ed-6378a1b63bc0 admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:47:57.403142 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8d503a-d056-4e6e-97ed-6378a1b63bc0 admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:47:57.407936 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbe1a6b-ff2b-4be2-961a-760045c3d40f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:47:57.408568 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbe1a6b-ff2b-4be2-961a-760045c3d40f admin admin] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:47:57.408911 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbe1a6b-ff2b-4be2-961a-760045c3d40f admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:47:57.409186 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbe1a6b-ff2b-4be2-961a-760045c3d40f admin admin] PATH_INFO: `/v3/projects` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:47:57.409953 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbe1a6b-ff2b-4be2-961a-760045c3d40f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:47:57.411888 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbe1a6b-ff2b-4be2-961a-760045c3d40f admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:47:57.518203 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1186/2370] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Aug 24 06:47:57 2022] POST /identity/v3/projects => generated 373 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:47:57.522268 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9029523-4b81-4a7f-bcc4-4838b26ff14e None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:47:57.533138 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9029523-4b81-4a7f-bcc4-4838b26ff14e None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:47:57.545813 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:47:57.590756 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613ce532-9ee4-46a2-a2ba-c81bf18c01d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:47:57.592253 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613ce532-9ee4-46a2-a2ba-c81bf18c01d9 admin admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:47:57.592253 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613ce532-9ee4-46a2-a2ba-c81bf18c01d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:47:57.592253 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613ce532-9ee4-46a2-a2ba-c81bf18c01d9 admin admin] PATH_INFO: `/v3/users` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:47:57.592587 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613ce532-9ee4-46a2-a2ba-c81bf18c01d9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:47:57.594416 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613ce532-9ee4-46a2-a2ba-c81bf18c01d9 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:47:57.689194 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1185/2371] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Aug 24 06:47:57 2022] POST /identity/v3/users => generated 293 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:47:57.693397 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbe1a6b-ff2b-4be2-961a-760045c3d40f admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:47:57.704078 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbe1a6b-ff2b-4be2-961a-760045c3d40f admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:47:57.708922 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8069850-5ac9-49a7-90b9-34ef60368c4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:47:57.709741 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8069850-5ac9-49a7-90b9-34ef60368c4b admin admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:47:57.710441 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8069850-5ac9-49a7-90b9-34ef60368c4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:47:57.710441 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8069850-5ac9-49a7-90b9-34ef60368c4b admin admin] PATH_INFO: `/v3/roles` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:47:57.710870 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8069850-5ac9-49a7-90b9-34ef60368c4b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:47:57.712469 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8069850-5ac9-49a7-90b9-34ef60368c4b admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:47:57.723437 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1187/2372] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Aug 24 06:47:57 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:47:57.729622 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613ce532-9ee4-46a2-a2ba-c81bf18c01d9 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:47:57.740965 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613ce532-9ee4-46a2-a2ba-c81bf18c01d9 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:47:57.745531 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d26de9-80a4-4ca2-82e8-47523349bc51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:47:57.746193 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d26de9-80a4-4ca2-82e8-47523349bc51 admin admin] REQUEST_METHOD: `PUT` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:47:57.746460 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d26de9-80a4-4ca2-82e8-47523349bc51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:47:57.746731 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d26de9-80a4-4ca2-82e8-47523349bc51 admin admin] PATH_INFO: `/v3/projects/c46c353208d84c318eaeef5815bfc14f/users/26c7b26a429a4bf286c6a2232f701a38/roles/49e47cd157d743f6905f1db07ec878a7` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:47:57.761705 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d26de9-80a4-4ca2-82e8-47523349bc51 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c46c353208d84c318eaeef5815bfc14f, user_id=26c7b26a429a4bf286c6a2232f701a38, role_id=49e47cd157d743f6905f1db07ec878a7)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:47:57.764329 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d26de9-80a4-4ca2-82e8-47523349bc51 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:47:57.859442 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1186/2373] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Aug 24 06:47:57 2022] PUT /identity/v3/projects/c46c353208d84c318eaeef5815bfc14f/users/26c7b26a429a4bf286c6a2232f701a38/roles/49e47cd157d743f6905f1db07ec878a7 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:47:57.863916 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8069850-5ac9-49a7-90b9-34ef60368c4b admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:47:57.873761 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8069850-5ac9-49a7-90b9-34ef60368c4b admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:47:57.930426 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f56619-5e3e-434d-8752-7920b4793acc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:47:57.931019 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f56619-5e3e-434d-8752-7920b4793acc admin admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:47:57.931295 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f56619-5e3e-434d-8752-7920b4793acc admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:47:57.931564 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f56619-5e3e-434d-8752-7920b4793acc admin admin] PATH_INFO: `/v3/roles` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:47:57.932187 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f56619-5e3e-434d-8752-7920b4793acc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:47:57.933798 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f56619-5e3e-434d-8752-7920b4793acc admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:47:57.943547 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1188/2374] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Aug 24 06:47:57 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:47:57.947816 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d26de9-80a4-4ca2-82e8-47523349bc51 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:47:57.958041 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d26de9-80a4-4ca2-82e8-47523349bc51 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:47:57.962667 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bf012a-6352-4cc7-b0ce-e6059cad0aff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:47:57.963314 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bf012a-6352-4cc7-b0ce-e6059cad0aff admin admin] REQUEST_METHOD: `PUT` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:47:57.963592 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bf012a-6352-4cc7-b0ce-e6059cad0aff admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:47:57.963864 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bf012a-6352-4cc7-b0ce-e6059cad0aff admin admin] PATH_INFO: `/v3/projects/c46c353208d84c318eaeef5815bfc14f/users/26c7b26a429a4bf286c6a2232f701a38/roles/49e47cd157d743f6905f1db07ec878a7` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:47:57.966338 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5bf012a-6352-4cc7-b0ce-e6059cad0aff admin admin] RBAC: Authorizing `identity:create_grant(project_id=c46c353208d84c318eaeef5815bfc14f, user_id=26c7b26a429a4bf286c6a2232f701a38, role_id=49e47cd157d743f6905f1db07ec878a7)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:47:57.969506 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5bf012a-6352-4cc7-b0ce-e6059cad0aff admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:47:57.979418 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1187/2375] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Aug 24 06:47:57 2022] PUT /identity/v3/projects/c46c353208d84c318eaeef5815bfc14f/users/26c7b26a429a4bf286c6a2232f701a38/roles/49e47cd157d743f6905f1db07ec878a7 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:47:57.990019 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d54ef0-64a0-408c-a9c7-a81e9eb4a06b None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:47:57.990308 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d54ef0-64a0-408c-a9c7-a81e9eb4a06b None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:47:57.990582 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d54ef0-64a0-408c-a9c7-a81e9eb4a06b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:47:58.017818 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-e8d54ef0-64a0-408c-a9c7-a81e9eb4a06b None None] MFA Rules not processed for user `26c7b26a429a4bf286c6a2232f701a38`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:47:58.019038 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-e8d54ef0-64a0-408c-a9c7-a81e9eb4a06b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:47:58.028363 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:47:58.086496 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1189/2376] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:47:57 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:48:07.975847 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=36,keystone:INSERT=5 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:48:08.144891 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=24 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:48:36.947121 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bf012a-6352-4cc7-b0ce-e6059cad0aff admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:48:36.948837 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-d5bf012a-6352-4cc7-b0ce-e6059cad0aff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:48:37.022357 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bf012a-6352-4cc7-b0ce-e6059cad0aff admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:48:37.027985 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c4c533-00af-42fa-9441-5ec165dd9bce service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb6d3743340241f98cacb9e0d063298c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:48:37.032213 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c4c533-00af-42fa-9441-5ec165dd9bce service nova] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:48:37.032213 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c4c533-00af-42fa-9441-5ec165dd9bce service nova] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:48:37.032213 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c4c533-00af-42fa-9441-5ec165dd9bce service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:48:37.032213 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-b7c4c533-00af-42fa-9441-5ec165dd9bce service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:48:37.088745 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c4c533-00af-42fa-9441-5ec165dd9bce service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:48:37.090517 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c4c533-00af-42fa-9441-5ec165dd9bce service nova] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:48:37.100387 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1188/2377] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:48:36 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:48:43.077234 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d54ef0-64a0-408c-a9c7-a81e9eb4a06b None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:48:43.079346 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-e8d54ef0-64a0-408c-a9c7-a81e9eb4a06b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:48:43.155404 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d54ef0-64a0-408c-a9c7-a81e9eb4a06b None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:48:43.159355 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d28c77-ba08-4a37-9da8-32a11499f3ce service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c4def55ac0e48b29836bc5c64bc7402', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:48:43.159978 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d28c77-ba08-4a37-9da8-32a11499f3ce service cinder] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:48:43.160251 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d28c77-ba08-4a37-9da8-32a11499f3ce service cinder] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:48:43.160539 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d28c77-ba08-4a37-9da8-32a11499f3ce service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:48:43.162361 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-29d28c77-ba08-4a37-9da8-32a11499f3ce service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:48:43.227057 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d28c77-ba08-4a37-9da8-32a11499f3ce service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:48:43.228380 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d28c77-ba08-4a37-9da8-32a11499f3ce service cinder] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:48:43.238481 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1190/2378] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:48:43 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:48:44.057199 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c4c533-00af-42fa-9441-5ec165dd9bce service nova] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:48:44.067507 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c4c533-00af-42fa-9441-5ec165dd9bce service nova] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:48:44.071425 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb865f9d-b03a-455b-92a2-911476a3c744 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c4def55ac0e48b29836bc5c64bc7402', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:48:44.072064 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb865f9d-b03a-455b-92a2-911476a3c744 service cinder] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:48:44.072576 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb865f9d-b03a-455b-92a2-911476a3c744 service cinder] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:48:44.072728 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb865f9d-b03a-455b-92a2-911476a3c744 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:48:44.084970 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb865f9d-b03a-455b-92a2-911476a3c744 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:48:44.086468 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb865f9d-b03a-455b-92a2-911476a3c744 service cinder] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:48:44.097917 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1189/2379] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:48:44 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 06:48:44.253036 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d28c77-ba08-4a37-9da8-32a11499f3ce service cinder] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:48:44.254520 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-29d28c77-ba08-4a37-9da8-32a11499f3ce service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:48:44.328724 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d28c77-ba08-4a37-9da8-32a11499f3ce service cinder] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:48:44.333834 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07047bb-e8c8-4183-9ee4-21dfd1ed9d1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:48:44.334551 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07047bb-e8c8-4183-9ee4-21dfd1ed9d1a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:48:44.334947 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07047bb-e8c8-4183-9ee4-21dfd1ed9d1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:48:44.335289 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07047bb-e8c8-4183-9ee4-21dfd1ed9d1a admin admin] PATH_INFO: `/v3/users/c8f4564533c846b49d46fc26da876c9b` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:48:44.337998 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07047bb-e8c8-4183-9ee4-21dfd1ed9d1a admin admin] RBAC: Authorizing `identity:delete_user(user_id=c8f4564533c846b49d46fc26da876c9b)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:48:44.339939 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07047bb-e8c8-4183-9ee4-21dfd1ed9d1a admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:48:44.585154 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:48:44.588291 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-c07047bb-e8c8-4183-9ee4-21dfd1ed9d1a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c8f4564533c846b49d46fc26da876c9b'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:48:44.596035 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-c07047bb-e8c8-4183-9ee4-21dfd1ed9d1a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c8f4564533c846b49d46fc26da876c9b'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:48:44.596766 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-c07047bb-e8c8-4183-9ee4-21dfd1ed9d1a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c8f4564533c846b49d46fc26da876c9b'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:48:44.597370 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-c07047bb-e8c8-4183-9ee4-21dfd1ed9d1a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c8f4564533c846b49d46fc26da876c9b'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:48:44.715815 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-c07047bb-e8c8-4183-9ee4-21dfd1ed9d1a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c8f4564533c846b49d46fc26da876c9b'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:48:44.734842 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1191/2380] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Aug 24 06:48:44 2022] DELETE /identity/v3/users/c8f4564533c846b49d46fc26da876c9b => generated 0 bytes in 482 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:48:44.739355 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cf9087-0d56-4bce-9de0-02408164926d None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:48:44.739644 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cf9087-0d56-4bce-9de0-02408164926d None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:48:44.739922 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cf9087-0d56-4bce-9de0-02408164926d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:48:44.759502 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-e7cf9087-0d56-4bce-9de0-02408164926d None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:48:44.760787 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-e7cf9087-0d56-4bce-9de0-02408164926d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:48:44.820538 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1190/2381] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:48:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:48:44.830495 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07047bb-e8c8-4183-9ee4-21dfd1ed9d1a admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:48:44.831914 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-c07047bb-e8c8-4183-9ee4-21dfd1ed9d1a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:48:44.888829 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07047bb-e8c8-4183-9ee4-21dfd1ed9d1a admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:48:44.892414 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7aa20a-fb5e-475b-a3a6-c548f0e64e6e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bb6c79db75e4d169331c02133d1ec32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:48:44.893026 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7aa20a-fb5e-475b-a3a6-c548f0e64e6e service neutron] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:48:44.893307 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7aa20a-fb5e-475b-a3a6-c548f0e64e6e service neutron] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:48:44.893606 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7aa20a-fb5e-475b-a3a6-c548f0e64e6e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:48:44.902625 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7aa20a-fb5e-475b-a3a6-c548f0e64e6e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:48:44.904078 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7aa20a-fb5e-475b-a3a6-c548f0e64e6e service neutron] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:48:44.913661 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1192/2382] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:48:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:48:45.352307 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cf9087-0d56-4bce-9de0-02408164926d None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:48:45.353818 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-e7cf9087-0d56-4bce-9de0-02408164926d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:48:45.366986 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cf9087-0d56-4bce-9de0-02408164926d None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:48:45.371639 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22de0b04-74b4-4377-84a8-d51e5a186156 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:48:45.372273 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22de0b04-74b4-4377-84a8-d51e5a186156 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:48:45.372579 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22de0b04-74b4-4377-84a8-d51e5a186156 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:48:45.372851 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22de0b04-74b4-4377-84a8-d51e5a186156 admin admin] PATH_INFO: `/v3/projects/89e7de2d11ec4e86b293a92de4e106b5` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:48:45.373996 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22de0b04-74b4-4377-84a8-d51e5a186156 admin admin] RBAC: Authorizing `identity:delete_project(project_id=89e7de2d11ec4e86b293a92de4e106b5)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:48:45.375902 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22de0b04-74b4-4377-84a8-d51e5a186156 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:48:45.399343 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:48:45.554038 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-22de0b04-74b4-4377-84a8-d51e5a186156 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '89e7de2d11ec4e86b293a92de4e106b5', 'request_id': 'req-22de0b04-74b4-4377-84a8-d51e5a186156'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:48:45.559595 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-22de0b04-74b4-4377-84a8-d51e5a186156 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '89e7de2d11ec4e86b293a92de4e106b5', 'request_id': 'req-22de0b04-74b4-4377-84a8-d51e5a186156'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:48:45.566086 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-22de0b04-74b4-4377-84a8-d51e5a186156 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '89e7de2d11ec4e86b293a92de4e106b5', 'request_id': 'req-22de0b04-74b4-4377-84a8-d51e5a186156'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:48:45.726838 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-22de0b04-74b4-4377-84a8-d51e5a186156 admin admin] The token cache is being invalidate because project 89e7de2d11ec4e86b293a92de4e106b5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81852) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:48:45.727247 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-22de0b04-74b4-4377-84a8-d51e5a186156 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:48:45.727907 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-22de0b04-74b4-4377-84a8-d51e5a186156 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:48:45.729505 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1191/2383] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Aug 24 06:48:45 2022] DELETE /identity/v3/projects/89e7de2d11ec4e86b293a92de4e106b5 => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:48:45.739580 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7aa20a-fb5e-475b-a3a6-c548f0e64e6e service neutron] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:48:45.743116 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7aa20a-fb5e-475b-a3a6-c548f0e64e6e service neutron] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:48:45.799177 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fa4c66-c648-4f8b-9928-b16160d455fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:48:45.799833 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fa4c66-c648-4f8b-9928-b16160d455fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:48:45.800114 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fa4c66-c648-4f8b-9928-b16160d455fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:48:45.800388 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fa4c66-c648-4f8b-9928-b16160d455fb admin admin] PATH_INFO: `/v3/users/389e6edbab16426788696806b2419165` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:48:45.801564 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fa4c66-c648-4f8b-9928-b16160d455fb admin admin] RBAC: Authorizing `identity:delete_user(user_id=389e6edbab16426788696806b2419165)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:48:45.803548 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fa4c66-c648-4f8b-9928-b16160d455fb admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:48:46.078440 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-61fa4c66-c648-4f8b-9928-b16160d455fb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '389e6edbab16426788696806b2419165'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:48:46.085756 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-61fa4c66-c648-4f8b-9928-b16160d455fb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '389e6edbab16426788696806b2419165'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:48:46.097918 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-61fa4c66-c648-4f8b-9928-b16160d455fb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '389e6edbab16426788696806b2419165'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:48:46.097918 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-61fa4c66-c648-4f8b-9928-b16160d455fb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '389e6edbab16426788696806b2419165'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:48:46.201438 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-61fa4c66-c648-4f8b-9928-b16160d455fb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '389e6edbab16426788696806b2419165'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:48:46.203789 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:48:46.219512 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1193/2384] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Aug 24 06:48:45 2022] DELETE /identity/v3/users/389e6edbab16426788696806b2419165 => generated 0 bytes in 487 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:48:46.706073 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22de0b04-74b4-4377-84a8-d51e5a186156 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:48:46.707721 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-22de0b04-74b4-4377-84a8-d51e5a186156 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:48:46.723494 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22de0b04-74b4-4377-84a8-d51e5a186156 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:48:46.782656 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c636d16-9d79-4060-a76c-4c3537ce7bd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:48:46.783222 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c636d16-9d79-4060-a76c-4c3537ce7bd1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:48:46.783501 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c636d16-9d79-4060-a76c-4c3537ce7bd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:48:46.783773 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c636d16-9d79-4060-a76c-4c3537ce7bd1 admin admin] PATH_INFO: `/v3/projects/c7704cd752bb48508f5540c94cd42ee6` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:48:46.784917 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c636d16-9d79-4060-a76c-4c3537ce7bd1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c7704cd752bb48508f5540c94cd42ee6)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:48:46.786840 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c636d16-9d79-4060-a76c-4c3537ce7bd1 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:48:46.944935 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-5c636d16-9d79-4060-a76c-4c3537ce7bd1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c7704cd752bb48508f5540c94cd42ee6', 'request_id': 'req-5c636d16-9d79-4060-a76c-4c3537ce7bd1'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:48:46.951006 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-5c636d16-9d79-4060-a76c-4c3537ce7bd1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c7704cd752bb48508f5540c94cd42ee6', 'request_id': 'req-5c636d16-9d79-4060-a76c-4c3537ce7bd1'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:48:46.957856 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-5c636d16-9d79-4060-a76c-4c3537ce7bd1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c7704cd752bb48508f5540c94cd42ee6', 'request_id': 'req-5c636d16-9d79-4060-a76c-4c3537ce7bd1'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:48:47.079582 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-5c636d16-9d79-4060-a76c-4c3537ce7bd1 admin admin] The token cache is being invalidate because project c7704cd752bb48508f5540c94cd42ee6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81852) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:48:47.079873 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-5c636d16-9d79-4060-a76c-4c3537ce7bd1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:48:47.080563 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-5c636d16-9d79-4060-a76c-4c3537ce7bd1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:48:47.082197 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1192/2385] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Aug 24 06:48:46 2022] DELETE /identity/v3/projects/c7704cd752bb48508f5540c94cd42ee6 => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:48:47.088022 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe96d3a-d7d9-4390-95f1-51f18240f8d1 None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:48:47.088615 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe96d3a-d7d9-4390-95f1-51f18240f8d1 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:48:47.088615 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe96d3a-d7d9-4390-95f1-51f18240f8d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:48:47.109686 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-abe96d3a-d7d9-4390-95f1-51f18240f8d1 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:48:47.110905 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-abe96d3a-d7d9-4390-95f1-51f18240f8d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:48:47.169893 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1194/2386] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:48:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:48:47.177959 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b2aa3e-8ba2-4ceb-b3c6-deb1dd8ab26e None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:48:47.178245 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b2aa3e-8ba2-4ceb-b3c6-deb1dd8ab26e None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:48:47.178528 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b2aa3e-8ba2-4ceb-b3c6-deb1dd8ab26e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:48:47.198477 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-76b2aa3e-8ba2-4ceb-b3c6-deb1dd8ab26e None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:48:47.199757 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-76b2aa3e-8ba2-4ceb-b3c6-deb1dd8ab26e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:48:47.208343 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1193/2387] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:48:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 06:48:47.212798 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe96d3a-d7d9-4390-95f1-51f18240f8d1 None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:48:47.223096 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe96d3a-d7d9-4390-95f1-51f18240f8d1 None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:48:47.231045 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d998a39-fb21-4875-af6c-e8c1f17d9a30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:48:47.231045 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d998a39-fb21-4875-af6c-e8c1f17d9a30 admin admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:48:47.231045 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d998a39-fb21-4875-af6c-e8c1f17d9a30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:48:47.231045 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d998a39-fb21-4875-af6c-e8c1f17d9a30 admin admin] PATH_INFO: `/v3/domains` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:48:47.231482 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d998a39-fb21-4875-af6c-e8c1f17d9a30 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:48:47.231482 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d998a39-fb21-4875-af6c-e8c1f17d9a30 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:48:47.231482 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d998a39-fb21-4875-af6c-e8c1f17d9a30 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:48:47.243250 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1195/2388] 10.222.0.62 () {58 vars in 1153 bytes} [Wed Aug 24 06:48:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:48:47.247816 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b2aa3e-8ba2-4ceb-b3c6-deb1dd8ab26e None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:48:47.257336 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b2aa3e-8ba2-4ceb-b3c6-deb1dd8ab26e None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:48:47.262075 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c42881-63d2-41cd-8020-0fe2579fe255 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:48:47.262710 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c42881-63d2-41cd-8020-0fe2579fe255 admin admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:48:47.262991 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c42881-63d2-41cd-8020-0fe2579fe255 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:48:47.263288 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c42881-63d2-41cd-8020-0fe2579fe255 admin admin] PATH_INFO: `/v3/projects` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:48:47.263952 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c42881-63d2-41cd-8020-0fe2579fe255 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:48:47.266475 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c42881-63d2-41cd-8020-0fe2579fe255 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:48:47.366847 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1194/2389] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Aug 24 06:48:47 2022] POST /identity/v3/projects => generated 369 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:48:47.371380 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d998a39-fb21-4875-af6c-e8c1f17d9a30 admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:48:47.382249 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d998a39-fb21-4875-af6c-e8c1f17d9a30 admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:48:47.439811 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14b4f0f-ccb6-4eea-89fb-d630b0872351 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:48:47.440421 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14b4f0f-ccb6-4eea-89fb-d630b0872351 admin admin] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:48:47.440744 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14b4f0f-ccb6-4eea-89fb-d630b0872351 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:48:47.441008 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14b4f0f-ccb6-4eea-89fb-d630b0872351 admin admin] PATH_INFO: `/v3/users` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:48:47.441651 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14b4f0f-ccb6-4eea-89fb-d630b0872351 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:48:47.443709 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14b4f0f-ccb6-4eea-89fb-d630b0872351 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:48:47.536258 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1196/2390] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Aug 24 06:48:47 2022] POST /identity/v3/users => generated 285 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:48:47.540389 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c42881-63d2-41cd-8020-0fe2579fe255 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:48:47.550789 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c42881-63d2-41cd-8020-0fe2579fe255 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:48:47.555390 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b361e48-2ecc-45ac-8b9d-09a97875f5dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:48:47.555984 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b361e48-2ecc-45ac-8b9d-09a97875f5dc admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:48:47.556263 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b361e48-2ecc-45ac-8b9d-09a97875f5dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:48:47.556569 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b361e48-2ecc-45ac-8b9d-09a97875f5dc admin admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:48:47.557214 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b361e48-2ecc-45ac-8b9d-09a97875f5dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:48:47.558831 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b361e48-2ecc-45ac-8b9d-09a97875f5dc admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:48:47.561200 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:48:47.570045 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1195/2391] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Aug 24 06:48:47 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:48:47.574047 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14b4f0f-ccb6-4eea-89fb-d630b0872351 admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:48:47.584777 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14b4f0f-ccb6-4eea-89fb-d630b0872351 admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:48:47.589406 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5b5a5e-45ce-4263-ac59-781bc7650e64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:48:47.589996 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5b5a5e-45ce-4263-ac59-781bc7650e64 admin admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:48:47.590276 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5b5a5e-45ce-4263-ac59-781bc7650e64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:48:47.590546 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5b5a5e-45ce-4263-ac59-781bc7650e64 admin admin] PATH_INFO: `/v3/roles` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:48:47.591256 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5b5a5e-45ce-4263-ac59-781bc7650e64 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:48:47.592896 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5b5a5e-45ce-4263-ac59-781bc7650e64 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:48:47.603965 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1197/2392] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Aug 24 06:48:47 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:48:47.614409 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b361e48-2ecc-45ac-8b9d-09a97875f5dc admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:48:47.624889 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b361e48-2ecc-45ac-8b9d-09a97875f5dc admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:48:47.629519 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8169e70c-ce14-4d10-9301-9b10fef823a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:48:47.630173 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8169e70c-ce14-4d10-9301-9b10fef823a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:48:47.630449 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8169e70c-ce14-4d10-9301-9b10fef823a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:48:47.630719 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8169e70c-ce14-4d10-9301-9b10fef823a2 admin admin] PATH_INFO: `/v3/projects/4a506cfdbf2b45e4abcd29b4fdec4625/users/f7092d51b8234bf0aced8af4034b759b/roles/580aa7dbb38d4cff8f9d1bb8983f99ef` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:48:47.646537 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8169e70c-ce14-4d10-9301-9b10fef823a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a506cfdbf2b45e4abcd29b4fdec4625, user_id=f7092d51b8234bf0aced8af4034b759b, role_id=580aa7dbb38d4cff8f9d1bb8983f99ef)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:48:47.649233 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8169e70c-ce14-4d10-9301-9b10fef823a2 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:48:47.773547 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1196/2393] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Aug 24 06:48:47 2022] PUT /identity/v3/projects/4a506cfdbf2b45e4abcd29b4fdec4625/users/f7092d51b8234bf0aced8af4034b759b/roles/580aa7dbb38d4cff8f9d1bb8983f99ef => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 06:48:47.785520 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f41b35-3bfc-42ea-ad77-6130688ffb7d None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:48:47.785788 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f41b35-3bfc-42ea-ad77-6130688ffb7d None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:48:47.786082 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f41b35-3bfc-42ea-ad77-6130688ffb7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:48:47.817867 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-94f41b35-3bfc-42ea-ad77-6130688ffb7d None None] MFA Rules not processed for user `f7092d51b8234bf0aced8af4034b759b`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:48:47.819005 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-94f41b35-3bfc-42ea-ad77-6130688ffb7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:48:47.862930 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:48:47.890641 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1198/2394] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:48:47 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:48:47.895367 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8169e70c-ce14-4d10-9301-9b10fef823a2 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:48:47.962392 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8169e70c-ce14-4d10-9301-9b10fef823a2 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:48:47.967125 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98dfb1eb-b85b-49c4-b853-5c2e709128a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:48:47.967730 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98dfb1eb-b85b-49c4-b853-5c2e709128a9 admin admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:48:47.968036 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98dfb1eb-b85b-49c4-b853-5c2e709128a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:48:47.968309 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98dfb1eb-b85b-49c4-b853-5c2e709128a9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:48:47.968983 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98dfb1eb-b85b-49c4-b853-5c2e709128a9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:48:47.970891 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98dfb1eb-b85b-49c4-b853-5c2e709128a9 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:48:48.088908 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1197/2395] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Aug 24 06:48:47 2022] POST /identity/v3/projects => generated 369 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:48:48.092841 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f41b35-3bfc-42ea-ad77-6130688ffb7d None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:48:48.104285 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f41b35-3bfc-42ea-ad77-6130688ffb7d None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:48:48.167026 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4880f7bc-7222-41a0-9df7-edaa936cd474 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:48:48.168613 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4880f7bc-7222-41a0-9df7-edaa936cd474 admin admin] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:48:48.168919 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4880f7bc-7222-41a0-9df7-edaa936cd474 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:48:48.169186 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4880f7bc-7222-41a0-9df7-edaa936cd474 admin admin] PATH_INFO: `/v3/users` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:48:48.169825 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4880f7bc-7222-41a0-9df7-edaa936cd474 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:48:48.172743 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4880f7bc-7222-41a0-9df7-edaa936cd474 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:48:48.282333 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1199/2396] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Aug 24 06:48:48 2022] POST /identity/v3/users => generated 291 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:48:48.286913 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98dfb1eb-b85b-49c4-b853-5c2e709128a9 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:48:48.298143 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98dfb1eb-b85b-49c4-b853-5c2e709128a9 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:48:48.302915 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa17adf-94c3-414d-83c0-de09fbf8436a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:48:48.303528 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa17adf-94c3-414d-83c0-de09fbf8436a admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:48:48.303804 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa17adf-94c3-414d-83c0-de09fbf8436a admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:48:48.304075 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa17adf-94c3-414d-83c0-de09fbf8436a admin admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:48:48.304748 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa17adf-94c3-414d-83c0-de09fbf8436a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:48:48.306356 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa17adf-94c3-414d-83c0-de09fbf8436a admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:48:48.317850 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1198/2397] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Aug 24 06:48:48 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:48:48.328036 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4880f7bc-7222-41a0-9df7-edaa936cd474 admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:48:48.342336 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4880f7bc-7222-41a0-9df7-edaa936cd474 admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:48:48.349505 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48385b9f-6589-4897-88b9-2d950f91ac10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:48:48.350285 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48385b9f-6589-4897-88b9-2d950f91ac10 admin admin] REQUEST_METHOD: `PUT` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:48:48.350721 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48385b9f-6589-4897-88b9-2d950f91ac10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:48:48.351184 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48385b9f-6589-4897-88b9-2d950f91ac10 admin admin] PATH_INFO: `/v3/projects/7c468f9d6a6f43b0885b1d76feb1add9/users/f1e144328fdd4ddcbf583e2a41619963/roles/49e47cd157d743f6905f1db07ec878a7` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:48:48.370273 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48385b9f-6589-4897-88b9-2d950f91ac10 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7c468f9d6a6f43b0885b1d76feb1add9, user_id=f1e144328fdd4ddcbf583e2a41619963, role_id=49e47cd157d743f6905f1db07ec878a7)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:48:48.373044 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48385b9f-6589-4897-88b9-2d950f91ac10 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:48:48.482209 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1200/2398] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Aug 24 06:48:48 2022] PUT /identity/v3/projects/7c468f9d6a6f43b0885b1d76feb1add9/users/f1e144328fdd4ddcbf583e2a41619963/roles/49e47cd157d743f6905f1db07ec878a7 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:48:48.486479 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa17adf-94c3-414d-83c0-de09fbf8436a admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:48:48.499881 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa17adf-94c3-414d-83c0-de09fbf8436a admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:48:48.561128 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55a92a0-a294-4310-a335-703cd7d000ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:48:48.561771 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55a92a0-a294-4310-a335-703cd7d000ff admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:48:48.562257 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55a92a0-a294-4310-a335-703cd7d000ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:48:48.562532 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55a92a0-a294-4310-a335-703cd7d000ff admin admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:48:48.563170 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55a92a0-a294-4310-a335-703cd7d000ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:48:48.564795 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55a92a0-a294-4310-a335-703cd7d000ff admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:48:48.575572 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1199/2399] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Aug 24 06:48:48 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:48:48.580460 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48385b9f-6589-4897-88b9-2d950f91ac10 admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:48:48.591077 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48385b9f-6589-4897-88b9-2d950f91ac10 admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:48:48.596694 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d36604-0668-44e0-9c9e-fb4e6fe1e26f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:48:48.597292 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d36604-0668-44e0-9c9e-fb4e6fe1e26f admin admin] REQUEST_METHOD: `PUT` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:48:48.597568 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d36604-0668-44e0-9c9e-fb4e6fe1e26f admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:48:48.597840 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d36604-0668-44e0-9c9e-fb4e6fe1e26f admin admin] PATH_INFO: `/v3/projects/7c468f9d6a6f43b0885b1d76feb1add9/users/f1e144328fdd4ddcbf583e2a41619963/roles/49e47cd157d743f6905f1db07ec878a7` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:48:48.599749 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d36604-0668-44e0-9c9e-fb4e6fe1e26f admin admin] RBAC: Authorizing `identity:create_grant(project_id=7c468f9d6a6f43b0885b1d76feb1add9, user_id=f1e144328fdd4ddcbf583e2a41619963, role_id=49e47cd157d743f6905f1db07ec878a7)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:48:48.602450 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d36604-0668-44e0-9c9e-fb4e6fe1e26f admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:48:48.613975 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1201/2400] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Aug 24 06:48:48 2022] PUT /identity/v3/projects/7c468f9d6a6f43b0885b1d76feb1add9/users/f1e144328fdd4ddcbf583e2a41619963/roles/49e47cd157d743f6905f1db07ec878a7 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:48:48.624644 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa502d1-41d9-4143-8bee-6bcd9200c199 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:48:48.624930 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa502d1-41d9-4143-8bee-6bcd9200c199 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:48:48.625269 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa502d1-41d9-4143-8bee-6bcd9200c199 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:48:48.659496 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-6aa502d1-41d9-4143-8bee-6bcd9200c199 None None] MFA Rules not processed for user `f1e144328fdd4ddcbf583e2a41619963`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:48:48.660832 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-6aa502d1-41d9-4143-8bee-6bcd9200c199 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:48:48.697940 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:48:48.736964 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1200/2401] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:48:48 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:48:58.609761 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=54,keystone:INSERT=5 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:48:58.943780 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=14 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:49:37.833605 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d36604-0668-44e0-9c9e-fb4e6fe1e26f admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:49:37.835575 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-e1d36604-0668-44e0-9c9e-fb4e6fe1e26f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:49:37.895645 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d36604-0668-44e0-9c9e-fb4e6fe1e26f admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:49:37.899125 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74587b6f-63cd-449c-9b47-46f632ff98f0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c4def55ac0e48b29836bc5c64bc7402', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:49:37.899724 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74587b6f-63cd-449c-9b47-46f632ff98f0 service cinder] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:49:37.899996 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74587b6f-63cd-449c-9b47-46f632ff98f0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:49:37.900263 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74587b6f-63cd-449c-9b47-46f632ff98f0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:49:37.909753 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74587b6f-63cd-449c-9b47-46f632ff98f0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:49:37.911177 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74587b6f-63cd-449c-9b47-46f632ff98f0 service cinder] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:49:37.920835 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1202/2402] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:49:37 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:49:47.916437 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=26 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:50:44.510005 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa502d1-41d9-4143-8bee-6bcd9200c199 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:44.511798 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-6aa502d1-41d9-4143-8bee-6bcd9200c199 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:50:44.525333 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa502d1-41d9-4143-8bee-6bcd9200c199 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:44.529170 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5189e9c0-39d0-402b-9b07-8b654894486d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c4def55ac0e48b29836bc5c64bc7402', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:44.529782 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5189e9c0-39d0-402b-9b07-8b654894486d service cinder] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:44.530061 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5189e9c0-39d0-402b-9b07-8b654894486d service cinder] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:44.530335 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5189e9c0-39d0-402b-9b07-8b654894486d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:44.532097 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-5189e9c0-39d0-402b-9b07-8b654894486d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:50:44.619869 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5189e9c0-39d0-402b-9b07-8b654894486d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:44.621253 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5189e9c0-39d0-402b-9b07-8b654894486d service cinder] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:44.640896 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1201/2403] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:50:44 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:50:44.664255 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74587b6f-63cd-449c-9b47-46f632ff98f0 service cinder] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:44.674154 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74587b6f-63cd-449c-9b47-46f632ff98f0 service cinder] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:44.677872 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84945fc1-6fb3-459e-9ffc-0cda555d1b61 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c4def55ac0e48b29836bc5c64bc7402', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:44.678539 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84945fc1-6fb3-459e-9ffc-0cda555d1b61 service cinder] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:44.678873 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84945fc1-6fb3-459e-9ffc-0cda555d1b61 service cinder] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:44.679180 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84945fc1-6fb3-459e-9ffc-0cda555d1b61 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:44.741979 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84945fc1-6fb3-459e-9ffc-0cda555d1b61 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:44.743440 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84945fc1-6fb3-459e-9ffc-0cda555d1b61 service cinder] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:44.753649 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1203/2404] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:50:44 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:50:45.710893 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5189e9c0-39d0-402b-9b07-8b654894486d service cinder] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:45.710893 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-5189e9c0-39d0-402b-9b07-8b654894486d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:50:45.789405 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5189e9c0-39d0-402b-9b07-8b654894486d service cinder] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:45.794202 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdeed7bc-ba3d-4c4b-bd32-c612061b7d5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:45.794843 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdeed7bc-ba3d-4c4b-bd32-c612061b7d5f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:45.795121 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdeed7bc-ba3d-4c4b-bd32-c612061b7d5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:45.795394 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdeed7bc-ba3d-4c4b-bd32-c612061b7d5f admin admin] PATH_INFO: `/v3/users/6aeae9bc6673480daa9c74c915b11a1d` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:45.796578 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdeed7bc-ba3d-4c4b-bd32-c612061b7d5f admin admin] RBAC: Authorizing `identity:delete_user(user_id=6aeae9bc6673480daa9c74c915b11a1d)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:45.798521 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdeed7bc-ba3d-4c4b-bd32-c612061b7d5f admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:46.104090 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-bdeed7bc-ba3d-4c4b-bd32-c612061b7d5f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6aeae9bc6673480daa9c74c915b11a1d'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:46.111753 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-bdeed7bc-ba3d-4c4b-bd32-c612061b7d5f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6aeae9bc6673480daa9c74c915b11a1d'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:46.112478 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-bdeed7bc-ba3d-4c4b-bd32-c612061b7d5f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6aeae9bc6673480daa9c74c915b11a1d'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:46.207705 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-bdeed7bc-ba3d-4c4b-bd32-c612061b7d5f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6aeae9bc6673480daa9c74c915b11a1d'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:46.219298 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:50:46.222262 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-bdeed7bc-ba3d-4c4b-bd32-c612061b7d5f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6aeae9bc6673480daa9c74c915b11a1d'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:46.229245 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1202/2405] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Aug 24 06:50:45 2022] DELETE /identity/v3/users/6aeae9bc6673480daa9c74c915b11a1d => generated 0 bytes in 525 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:50:46.234386 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722eadc2-fa04-4671-a60e-377ec9c545f6 None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:46.234936 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722eadc2-fa04-4671-a60e-377ec9c545f6 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:46.234936 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722eadc2-fa04-4671-a60e-377ec9c545f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:46.255731 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-722eadc2-fa04-4671-a60e-377ec9c545f6 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:50:46.257000 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-722eadc2-fa04-4671-a60e-377ec9c545f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:50:46.322927 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1204/2406] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:50:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 06:50:46.334674 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdeed7bc-ba3d-4c4b-bd32-c612061b7d5f admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:46.336170 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-bdeed7bc-ba3d-4c4b-bd32-c612061b7d5f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:50:46.399420 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdeed7bc-ba3d-4c4b-bd32-c612061b7d5f admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:46.403193 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716f2d09-264d-48e1-8f60-9a9967051917 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bb6c79db75e4d169331c02133d1ec32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:46.403858 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716f2d09-264d-48e1-8f60-9a9967051917 service neutron] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:46.404140 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716f2d09-264d-48e1-8f60-9a9967051917 service neutron] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:46.404414 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716f2d09-264d-48e1-8f60-9a9967051917 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:46.414585 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716f2d09-264d-48e1-8f60-9a9967051917 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:46.416370 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716f2d09-264d-48e1-8f60-9a9967051917 service neutron] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:46.427160 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1203/2407] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:50:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:50:46.897596 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722eadc2-fa04-4671-a60e-377ec9c545f6 None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:46.899044 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-722eadc2-fa04-4671-a60e-377ec9c545f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:50:46.916589 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722eadc2-fa04-4671-a60e-377ec9c545f6 None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:46.924747 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28db7ac-f557-4481-a6e2-7f33545e267a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:46.925793 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28db7ac-f557-4481-a6e2-7f33545e267a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:46.926279 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28db7ac-f557-4481-a6e2-7f33545e267a admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:46.926755 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28db7ac-f557-4481-a6e2-7f33545e267a admin admin] PATH_INFO: `/v3/projects/b3d00da71a4447d38459b39369bde7bc` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:46.929903 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28db7ac-f557-4481-a6e2-7f33545e267a admin admin] RBAC: Authorizing `identity:delete_project(project_id=b3d00da71a4447d38459b39369bde7bc)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:46.932010 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28db7ac-f557-4481-a6e2-7f33545e267a admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:47.098911 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-d28db7ac-f557-4481-a6e2-7f33545e267a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b3d00da71a4447d38459b39369bde7bc', 'request_id': 'req-d28db7ac-f557-4481-a6e2-7f33545e267a'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:47.104834 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-d28db7ac-f557-4481-a6e2-7f33545e267a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b3d00da71a4447d38459b39369bde7bc', 'request_id': 'req-d28db7ac-f557-4481-a6e2-7f33545e267a'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:47.111662 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-d28db7ac-f557-4481-a6e2-7f33545e267a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b3d00da71a4447d38459b39369bde7bc', 'request_id': 'req-d28db7ac-f557-4481-a6e2-7f33545e267a'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:47.245947 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-d28db7ac-f557-4481-a6e2-7f33545e267a admin admin] The token cache is being invalidate because project b3d00da71a4447d38459b39369bde7bc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81851) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:50:47.246654 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-d28db7ac-f557-4481-a6e2-7f33545e267a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:47.246993 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-d28db7ac-f557-4481-a6e2-7f33545e267a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:47.249139 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1205/2408] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Aug 24 06:50:46 2022] DELETE /identity/v3/projects/b3d00da71a4447d38459b39369bde7bc => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:50:47.255671 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716f2d09-264d-48e1-8f60-9a9967051917 service neutron] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:47.270306 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716f2d09-264d-48e1-8f60-9a9967051917 service neutron] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:47.330940 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55aa7a7-b43d-4795-a994-2ea162477a29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:47.331576 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55aa7a7-b43d-4795-a994-2ea162477a29 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:47.331875 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55aa7a7-b43d-4795-a994-2ea162477a29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:47.332122 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55aa7a7-b43d-4795-a994-2ea162477a29 admin admin] PATH_INFO: `/v3/users/26c7b26a429a4bf286c6a2232f701a38` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:47.333271 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55aa7a7-b43d-4795-a994-2ea162477a29 admin admin] RBAC: Authorizing `identity:delete_user(user_id=26c7b26a429a4bf286c6a2232f701a38)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:47.335204 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55aa7a7-b43d-4795-a994-2ea162477a29 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:47.597969 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-e55aa7a7-b43d-4795-a994-2ea162477a29 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '26c7b26a429a4bf286c6a2232f701a38'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:47.605528 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-e55aa7a7-b43d-4795-a994-2ea162477a29 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '26c7b26a429a4bf286c6a2232f701a38'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:47.606252 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-e55aa7a7-b43d-4795-a994-2ea162477a29 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '26c7b26a429a4bf286c6a2232f701a38'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:47.724442 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-e55aa7a7-b43d-4795-a994-2ea162477a29 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '26c7b26a429a4bf286c6a2232f701a38'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:47.738620 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-e55aa7a7-b43d-4795-a994-2ea162477a29 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '26c7b26a429a4bf286c6a2232f701a38'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:47.745251 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1204/2409] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Aug 24 06:50:47 2022] DELETE /identity/v3/users/26c7b26a429a4bf286c6a2232f701a38 => generated 0 bytes in 491 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:50:48.292177 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28db7ac-f557-4481-a6e2-7f33545e267a admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:48.293844 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-d28db7ac-f557-4481-a6e2-7f33545e267a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:50:48.309439 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28db7ac-f557-4481-a6e2-7f33545e267a admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:48.332826 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:50:48.386329 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ff2b77-1e68-4eab-ae4a-4c5b624f7cc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:48.387148 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ff2b77-1e68-4eab-ae4a-4c5b624f7cc0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:48.387473 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ff2b77-1e68-4eab-ae4a-4c5b624f7cc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:48.388269 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ff2b77-1e68-4eab-ae4a-4c5b624f7cc0 admin admin] PATH_INFO: `/v3/projects/c46c353208d84c318eaeef5815bfc14f` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:48.389531 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ff2b77-1e68-4eab-ae4a-4c5b624f7cc0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c46c353208d84c318eaeef5815bfc14f)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:48.391678 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ff2b77-1e68-4eab-ae4a-4c5b624f7cc0 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:48.509256 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-00ff2b77-1e68-4eab-ae4a-4c5b624f7cc0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c46c353208d84c318eaeef5815bfc14f', 'request_id': 'req-00ff2b77-1e68-4eab-ae4a-4c5b624f7cc0'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:48.515342 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-00ff2b77-1e68-4eab-ae4a-4c5b624f7cc0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c46c353208d84c318eaeef5815bfc14f', 'request_id': 'req-00ff2b77-1e68-4eab-ae4a-4c5b624f7cc0'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:48.522744 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-00ff2b77-1e68-4eab-ae4a-4c5b624f7cc0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c46c353208d84c318eaeef5815bfc14f', 'request_id': 'req-00ff2b77-1e68-4eab-ae4a-4c5b624f7cc0'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:48.587224 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-00ff2b77-1e68-4eab-ae4a-4c5b624f7cc0 admin admin] The token cache is being invalidate because project c46c353208d84c318eaeef5815bfc14f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81851) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:50:48.587644 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-00ff2b77-1e68-4eab-ae4a-4c5b624f7cc0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:48.588300 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-00ff2b77-1e68-4eab-ae4a-4c5b624f7cc0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:48.589836 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1206/2410] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Aug 24 06:50:48 2022] DELETE /identity/v3/projects/c46c353208d84c318eaeef5815bfc14f => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 06:50:48.595620 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3693f0a-8baa-48a8-a2b5-ecc83535de51 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:48.596154 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3693f0a-8baa-48a8-a2b5-ecc83535de51 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:48.596154 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3693f0a-8baa-48a8-a2b5-ecc83535de51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:48.615496 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-b3693f0a-8baa-48a8-a2b5-ecc83535de51 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:50:48.616812 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-b3693f0a-8baa-48a8-a2b5-ecc83535de51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:50:48.622151 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:50:48.676789 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1205/2411] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:50:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:50:48.685851 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae5d4b3-bbf0-400c-8a30-d98c714f18a2 None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:48.686119 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae5d4b3-bbf0-400c-8a30-d98c714f18a2 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:48.686379 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae5d4b3-bbf0-400c-8a30-d98c714f18a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:48.707963 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-9ae5d4b3-bbf0-400c-8a30-d98c714f18a2 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:50:48.709266 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-9ae5d4b3-bbf0-400c-8a30-d98c714f18a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:50:48.717400 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1207/2412] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:50:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 06:50:48.721766 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3693f0a-8baa-48a8-a2b5-ecc83535de51 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:48.731840 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3693f0a-8baa-48a8-a2b5-ecc83535de51 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:48.736621 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec69ccf-2aae-4e21-a760-0224cbd84a88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:48.740978 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec69ccf-2aae-4e21-a760-0224cbd84a88 admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:48.740978 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec69ccf-2aae-4e21-a760-0224cbd84a88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:48.740978 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec69ccf-2aae-4e21-a760-0224cbd84a88 admin admin] PATH_INFO: `/v3/domains` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:48.740978 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec69ccf-2aae-4e21-a760-0224cbd84a88 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:50:48.740978 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec69ccf-2aae-4e21-a760-0224cbd84a88 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:48.740978 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec69ccf-2aae-4e21-a760-0224cbd84a88 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:48.753192 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1206/2413] 10.222.0.62 () {58 vars in 1153 bytes} [Wed Aug 24 06:50:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:50:48.758203 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae5d4b3-bbf0-400c-8a30-d98c714f18a2 None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:48.769299 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae5d4b3-bbf0-400c-8a30-d98c714f18a2 None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:48.774185 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da516de-6b24-49c7-8260-e1463b35055e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:48.774831 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da516de-6b24-49c7-8260-e1463b35055e admin admin] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:48.775109 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da516de-6b24-49c7-8260-e1463b35055e admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:48.775381 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da516de-6b24-49c7-8260-e1463b35055e admin admin] PATH_INFO: `/v3/projects` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:48.776020 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da516de-6b24-49c7-8260-e1463b35055e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:48.778096 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da516de-6b24-49c7-8260-e1463b35055e admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:48.875655 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1208/2414] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Aug 24 06:50:48 2022] POST /identity/v3/projects => generated 387 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:50:48.887060 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec69ccf-2aae-4e21-a760-0224cbd84a88 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:48.898115 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec69ccf-2aae-4e21-a760-0224cbd84a88 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:48.959571 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ab0fe8-6041-4114-9d17-55cccfa5fb77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:48.960353 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ab0fe8-6041-4114-9d17-55cccfa5fb77 admin admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:48.960713 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ab0fe8-6041-4114-9d17-55cccfa5fb77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:48.961042 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ab0fe8-6041-4114-9d17-55cccfa5fb77 admin admin] PATH_INFO: `/v3/users` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:48.961958 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ab0fe8-6041-4114-9d17-55cccfa5fb77 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:48.964381 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ab0fe8-6041-4114-9d17-55cccfa5fb77 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:49.096917 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1207/2415] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Aug 24 06:50:48 2022] POST /identity/v3/users => generated 294 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:50:49.101037 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da516de-6b24-49c7-8260-e1463b35055e admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:49.112159 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da516de-6b24-49c7-8260-e1463b35055e admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:49.116768 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acda58de-cba4-40fc-8e27-b3a6a9187305 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:49.117374 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acda58de-cba4-40fc-8e27-b3a6a9187305 admin admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:49.117649 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acda58de-cba4-40fc-8e27-b3a6a9187305 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:49.117910 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acda58de-cba4-40fc-8e27-b3a6a9187305 admin admin] PATH_INFO: `/v3/roles` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:49.118575 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acda58de-cba4-40fc-8e27-b3a6a9187305 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:49.120164 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acda58de-cba4-40fc-8e27-b3a6a9187305 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:49.131381 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1209/2416] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Aug 24 06:50:49 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:50:49.135855 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ab0fe8-6041-4114-9d17-55cccfa5fb77 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:49.147053 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ab0fe8-6041-4114-9d17-55cccfa5fb77 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:49.152204 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b01b07-6fa8-4cac-b38e-620ed0f178dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:49.152905 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b01b07-6fa8-4cac-b38e-620ed0f178dd admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:49.153199 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b01b07-6fa8-4cac-b38e-620ed0f178dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:49.153514 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b01b07-6fa8-4cac-b38e-620ed0f178dd admin admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:49.154214 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b01b07-6fa8-4cac-b38e-620ed0f178dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:49.155995 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b01b07-6fa8-4cac-b38e-620ed0f178dd admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:49.168049 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1208/2417] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Aug 24 06:50:49 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:50:49.173138 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acda58de-cba4-40fc-8e27-b3a6a9187305 admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:49.183636 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acda58de-cba4-40fc-8e27-b3a6a9187305 admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:49.189109 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986a5eab-2509-47bd-9b64-7b99919a5a04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:49.189765 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986a5eab-2509-47bd-9b64-7b99919a5a04 admin admin] REQUEST_METHOD: `PUT` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:49.190043 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986a5eab-2509-47bd-9b64-7b99919a5a04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:49.190313 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986a5eab-2509-47bd-9b64-7b99919a5a04 admin admin] PATH_INFO: `/v3/projects/58bc554786794fffb848e3db89b979e7/users/ac81d76770744c05aca25046fc00ebe2/roles/580aa7dbb38d4cff8f9d1bb8983f99ef` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:49.205577 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986a5eab-2509-47bd-9b64-7b99919a5a04 admin admin] RBAC: Authorizing `identity:create_grant(project_id=58bc554786794fffb848e3db89b979e7, user_id=ac81d76770744c05aca25046fc00ebe2, role_id=580aa7dbb38d4cff8f9d1bb8983f99ef)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:49.208242 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986a5eab-2509-47bd-9b64-7b99919a5a04 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:49.290716 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1210/2418] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Aug 24 06:50:49 2022] PUT /identity/v3/projects/58bc554786794fffb848e3db89b979e7/users/ac81d76770744c05aca25046fc00ebe2/roles/580aa7dbb38d4cff8f9d1bb8983f99ef => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:50:49.303989 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e6b3ec-6224-4aa6-ac48-28911c4a487d None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:49.304308 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e6b3ec-6224-4aa6-ac48-28911c4a487d None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:49.304641 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e6b3ec-6224-4aa6-ac48-28911c4a487d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:49.335458 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-a7e6b3ec-6224-4aa6-ac48-28911c4a487d None None] MFA Rules not processed for user `ac81d76770744c05aca25046fc00ebe2`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:50:49.336713 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-a7e6b3ec-6224-4aa6-ac48-28911c4a487d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:50:49.403666 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1209/2419] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:50:49 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:50:49.408444 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986a5eab-2509-47bd-9b64-7b99919a5a04 admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:49.459635 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:50:49.474852 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986a5eab-2509-47bd-9b64-7b99919a5a04 admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:49.482163 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477c1907-d729-4397-8aed-f548d6aa0ed5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:49.482812 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477c1907-d729-4397-8aed-f548d6aa0ed5 admin admin] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:49.483087 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477c1907-d729-4397-8aed-f548d6aa0ed5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:49.483356 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477c1907-d729-4397-8aed-f548d6aa0ed5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:49.483979 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477c1907-d729-4397-8aed-f548d6aa0ed5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:49.486162 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477c1907-d729-4397-8aed-f548d6aa0ed5 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:49.619456 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1211/2420] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Aug 24 06:50:49 2022] POST /identity/v3/projects => generated 387 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:50:49.623799 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e6b3ec-6224-4aa6-ac48-28911c4a487d None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:49.635254 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e6b3ec-6224-4aa6-ac48-28911c4a487d None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:49.639236 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:50:49.696084 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baaf087b-93f7-403b-a526-a64fb61e6bca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:49.696731 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaf087b-93f7-403b-a526-a64fb61e6bca admin admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:49.697008 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaf087b-93f7-403b-a526-a64fb61e6bca admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:49.697278 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaf087b-93f7-403b-a526-a64fb61e6bca admin admin] PATH_INFO: `/v3/users` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:49.697915 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baaf087b-93f7-403b-a526-a64fb61e6bca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:49.699863 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baaf087b-93f7-403b-a526-a64fb61e6bca admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:49.836689 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1210/2421] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Aug 24 06:50:49 2022] POST /identity/v3/users => generated 294 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:50:49.841572 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477c1907-d729-4397-8aed-f548d6aa0ed5 admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:49.852736 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477c1907-d729-4397-8aed-f548d6aa0ed5 admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:49.857501 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f951ea-96b4-4715-aae4-9f8577b484c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:49.858106 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f951ea-96b4-4715-aae4-9f8577b484c6 admin admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:49.858380 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f951ea-96b4-4715-aae4-9f8577b484c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:49.858646 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f951ea-96b4-4715-aae4-9f8577b484c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:49.859286 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f951ea-96b4-4715-aae4-9f8577b484c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:49.860927 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f951ea-96b4-4715-aae4-9f8577b484c6 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:49.871319 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1212/2422] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Aug 24 06:50:49 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:50:49.875549 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baaf087b-93f7-403b-a526-a64fb61e6bca admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:49.890850 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baaf087b-93f7-403b-a526-a64fb61e6bca admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:49.898376 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb56b8f-3c1d-4d57-9b13-297a02a54e66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:49.898376 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb56b8f-3c1d-4d57-9b13-297a02a54e66 admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:49.898376 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb56b8f-3c1d-4d57-9b13-297a02a54e66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:49.898823 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb56b8f-3c1d-4d57-9b13-297a02a54e66 admin admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:49.899518 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb56b8f-3c1d-4d57-9b13-297a02a54e66 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:49.901498 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb56b8f-3c1d-4d57-9b13-297a02a54e66 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:49.916274 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1211/2423] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Aug 24 06:50:49 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 06:50:49.920642 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f951ea-96b4-4715-aae4-9f8577b484c6 admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:49.931770 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f951ea-96b4-4715-aae4-9f8577b484c6 admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:49.937790 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9441b95-c3da-4f94-b11a-402838b1b649 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:49.938604 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9441b95-c3da-4f94-b11a-402838b1b649 admin admin] REQUEST_METHOD: `PUT` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:49.938892 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9441b95-c3da-4f94-b11a-402838b1b649 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:49.939168 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9441b95-c3da-4f94-b11a-402838b1b649 admin admin] PATH_INFO: `/v3/projects/82913b80c8224c7c9438272bc1ee11f2/users/b779cf208fb047659628ec97051ebeb9/roles/580aa7dbb38d4cff8f9d1bb8983f99ef` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:49.956699 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9441b95-c3da-4f94-b11a-402838b1b649 admin admin] RBAC: Authorizing `identity:create_grant(project_id=82913b80c8224c7c9438272bc1ee11f2, user_id=b779cf208fb047659628ec97051ebeb9, role_id=580aa7dbb38d4cff8f9d1bb8983f99ef)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:49.959434 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9441b95-c3da-4f94-b11a-402838b1b649 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:50.111187 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1213/2424] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Aug 24 06:50:49 2022] PUT /identity/v3/projects/82913b80c8224c7c9438272bc1ee11f2/users/b779cf208fb047659628ec97051ebeb9/roles/580aa7dbb38d4cff8f9d1bb8983f99ef => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:50:50.121836 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbf112e-2a90-4a77-916e-70aca6123bfc None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:50.122064 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbf112e-2a90-4a77-916e-70aca6123bfc None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:50.122312 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbf112e-2a90-4a77-916e-70aca6123bfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:50.151490 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-7cbf112e-2a90-4a77-916e-70aca6123bfc None None] MFA Rules not processed for user `b779cf208fb047659628ec97051ebeb9`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:50:50.152729 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-7cbf112e-2a90-4a77-916e-70aca6123bfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:50:50.220896 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1212/2425] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:50:50 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:50:50.227354 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9441b95-c3da-4f94-b11a-402838b1b649 admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:50.303626 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9441b95-c3da-4f94-b11a-402838b1b649 admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:50.308529 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444a8ff1-d545-4d07-9666-04c6873e5e96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:50.309148 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444a8ff1-d545-4d07-9666-04c6873e5e96 admin admin] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:50.309428 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444a8ff1-d545-4d07-9666-04c6873e5e96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:50.310477 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444a8ff1-d545-4d07-9666-04c6873e5e96 admin admin] PATH_INFO: `/v3/projects` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:50.311176 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444a8ff1-d545-4d07-9666-04c6873e5e96 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:50.313162 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444a8ff1-d545-4d07-9666-04c6873e5e96 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:50.417706 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1214/2426] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Aug 24 06:50:50 2022] POST /identity/v3/projects => generated 387 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:50:50.422351 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbf112e-2a90-4a77-916e-70aca6123bfc None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:50.434344 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbf112e-2a90-4a77-916e-70aca6123bfc None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:50.501789 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e44121b-5c60-4b60-b0c4-6689e1c75cef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:50.502415 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e44121b-5c60-4b60-b0c4-6689e1c75cef admin admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:50.502694 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e44121b-5c60-4b60-b0c4-6689e1c75cef admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:50.503053 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e44121b-5c60-4b60-b0c4-6689e1c75cef admin admin] PATH_INFO: `/v3/users` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:50.503620 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e44121b-5c60-4b60-b0c4-6689e1c75cef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:50.505657 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e44121b-5c60-4b60-b0c4-6689e1c75cef admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:50.624181 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1213/2427] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Aug 24 06:50:50 2022] POST /identity/v3/users => generated 300 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:50:50.628375 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444a8ff1-d545-4d07-9666-04c6873e5e96 admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:50.639208 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444a8ff1-d545-4d07-9666-04c6873e5e96 admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:50.644113 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444d44f3-6605-40e7-923a-3e1bcc090ce2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:50.644757 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444d44f3-6605-40e7-923a-3e1bcc090ce2 admin admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:50.645035 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444d44f3-6605-40e7-923a-3e1bcc090ce2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:50.645304 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444d44f3-6605-40e7-923a-3e1bcc090ce2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:50.645970 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444d44f3-6605-40e7-923a-3e1bcc090ce2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:50.647603 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444d44f3-6605-40e7-923a-3e1bcc090ce2 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:50.659066 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1215/2428] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Aug 24 06:50:50 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:50:50.663907 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e44121b-5c60-4b60-b0c4-6689e1c75cef admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:50.671410 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:50:50.675594 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e44121b-5c60-4b60-b0c4-6689e1c75cef admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:50.680327 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135cf153-8fe7-4f47-9992-8ce9774489b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:50.680982 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135cf153-8fe7-4f47-9992-8ce9774489b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:50.681257 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135cf153-8fe7-4f47-9992-8ce9774489b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:50.681531 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135cf153-8fe7-4f47-9992-8ce9774489b7 admin admin] PATH_INFO: `/v3/projects/b0157f47a7464a7d87f4005c6670fd81/users/f4e465232d5146019cea15f56494af22/roles/49e47cd157d743f6905f1db07ec878a7` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:50.699780 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135cf153-8fe7-4f47-9992-8ce9774489b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b0157f47a7464a7d87f4005c6670fd81, user_id=f4e465232d5146019cea15f56494af22, role_id=49e47cd157d743f6905f1db07ec878a7)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:50.702522 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135cf153-8fe7-4f47-9992-8ce9774489b7 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:50.776333 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1214/2429] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Aug 24 06:50:50 2022] PUT /identity/v3/projects/b0157f47a7464a7d87f4005c6670fd81/users/f4e465232d5146019cea15f56494af22/roles/49e47cd157d743f6905f1db07ec878a7 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 06:50:50.780713 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444d44f3-6605-40e7-923a-3e1bcc090ce2 admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:50.791630 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444d44f3-6605-40e7-923a-3e1bcc090ce2 admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:50.855746 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e83d92-8748-4a58-bee8-50349cb28a7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:50.856893 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e83d92-8748-4a58-bee8-50349cb28a7a admin admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:50.857193 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e83d92-8748-4a58-bee8-50349cb28a7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:50.857467 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e83d92-8748-4a58-bee8-50349cb28a7a admin admin] PATH_INFO: `/v3/roles` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:50.858112 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e83d92-8748-4a58-bee8-50349cb28a7a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:50.859780 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e83d92-8748-4a58-bee8-50349cb28a7a admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:50.870466 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1216/2430] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Aug 24 06:50:50 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 06:50:50.875125 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135cf153-8fe7-4f47-9992-8ce9774489b7 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:50.885982 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135cf153-8fe7-4f47-9992-8ce9774489b7 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:50.890668 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099dc851-a569-4d8b-9e84-994ab1de0f9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:50.891338 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099dc851-a569-4d8b-9e84-994ab1de0f9d admin admin] REQUEST_METHOD: `PUT` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:50.891611 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099dc851-a569-4d8b-9e84-994ab1de0f9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:50.891880 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099dc851-a569-4d8b-9e84-994ab1de0f9d admin admin] PATH_INFO: `/v3/projects/b0157f47a7464a7d87f4005c6670fd81/users/f4e465232d5146019cea15f56494af22/roles/49e47cd157d743f6905f1db07ec878a7` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:50.893844 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099dc851-a569-4d8b-9e84-994ab1de0f9d admin admin] RBAC: Authorizing `identity:create_grant(project_id=b0157f47a7464a7d87f4005c6670fd81, user_id=f4e465232d5146019cea15f56494af22, role_id=49e47cd157d743f6905f1db07ec878a7)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:50.896558 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099dc851-a569-4d8b-9e84-994ab1de0f9d admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:50.907489 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1215/2431] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Aug 24 06:50:50 2022] PUT /identity/v3/projects/b0157f47a7464a7d87f4005c6670fd81/users/f4e465232d5146019cea15f56494af22/roles/49e47cd157d743f6905f1db07ec878a7 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:50:50.919791 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842b66d3-752e-4e35-8eea-55c8283cd747 None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:50.920215 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842b66d3-752e-4e35-8eea-55c8283cd747 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:50.920807 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842b66d3-752e-4e35-8eea-55c8283cd747 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:50.945976 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:50:50.957168 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-842b66d3-752e-4e35-8eea-55c8283cd747 None None] MFA Rules not processed for user `f4e465232d5146019cea15f56494af22`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:50:50.958747 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-842b66d3-752e-4e35-8eea-55c8283cd747 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:50:51.042425 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1217/2432] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:50:50 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:51:00.903552 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=13,keystone:INSERT=2 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:51:01.098820 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=30 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:51:03.882764 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099dc851-a569-4d8b-9e84-994ab1de0f9d admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:03.884777 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-099dc851-a569-4d8b-9e84-994ab1de0f9d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:51:03.957364 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099dc851-a569-4d8b-9e84-994ab1de0f9d admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:03.962720 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7774439c-ee37-48d2-97da-830bad29bbc0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abe9d8c9291845dfa4ef55034e5c9e32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:03.963364 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7774439c-ee37-48d2-97da-830bad29bbc0 service placement] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:03.963646 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7774439c-ee37-48d2-97da-830bad29bbc0 service placement] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:03.963919 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7774439c-ee37-48d2-97da-830bad29bbc0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:03.965943 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-7774439c-ee37-48d2-97da-830bad29bbc0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:51:03.978843 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7774439c-ee37-48d2-97da-830bad29bbc0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:03.980565 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7774439c-ee37-48d2-97da-830bad29bbc0 service placement] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:03.992081 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1216/2433] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:51:03 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:51:13.987363 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=32 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:51:35.218188 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842b66d3-752e-4e35-8eea-55c8283cd747 None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:35.220338 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-842b66d3-752e-4e35-8eea-55c8283cd747 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:51:35.258897 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7774439c-ee37-48d2-97da-830bad29bbc0 service placement] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:35.260597 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-7774439c-ee37-48d2-97da-830bad29bbc0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:51:35.298420 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842b66d3-752e-4e35-8eea-55c8283cd747 None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:35.303306 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef2920c-15dc-476a-b7cb-89622e8a4530 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c4def55ac0e48b29836bc5c64bc7402', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:35.303997 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef2920c-15dc-476a-b7cb-89622e8a4530 service cinder] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:35.304302 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef2920c-15dc-476a-b7cb-89622e8a4530 service cinder] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:35.304600 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef2920c-15dc-476a-b7cb-89622e8a4530 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:35.314667 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef2920c-15dc-476a-b7cb-89622e8a4530 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:35.316105 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef2920c-15dc-476a-b7cb-89622e8a4530 service cinder] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:35.327104 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1218/2434] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:51:35 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:51:35.363803 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7774439c-ee37-48d2-97da-830bad29bbc0 service placement] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:35.367733 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ae867d-06e9-472b-af42-954629dc7b45 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7d52162cbdc4218b577ecc1e69a88c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:35.368271 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ae867d-06e9-472b-af42-954629dc7b45 service glance] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:35.368638 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ae867d-06e9-472b-af42-954629dc7b45 service glance] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:35.368911 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ae867d-06e9-472b-af42-954629dc7b45 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:35.370792 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-e4ae867d-06e9-472b-af42-954629dc7b45 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:51:35.404409 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application [None req-e4ae867d-06e9-472b-af42-954629dc7b45 service glance] Could not find project: df659b3447644e7dbe0f124b881e4344.: keystone.exception.ProjectNotFound: Could not find project: df659b3447644e7dbe0f124b881e4344. Aug 24 06:51:35.404409 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:51:35.404409 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:51:35.404409 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:51:35.404409 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:51:35.404409 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:51:35.404409 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:51:35.404409 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:51:35.404409 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:51:35.404409 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:51:35.404409 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:51:35.404409 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:51:35.404409 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/auth.py", line 285, in get Aug 24 06:51:35.404409 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:validate_token') Aug 24 06:51:35.404409 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 422, in enforce_call Aug 24 06:51:35.408595 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application subj_token_target_data = cls._extract_subject_token_target_data() Aug 24 06:51:35.408595 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 261, in _extract_subject_token_target_data Aug 24 06:51:35.408595 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application token = PROVIDER_APIS.token_provider_api.validate_token( Aug 24 06:51:35.408595 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:51:35.408595 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:51:35.408595 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 145, in validate_token Aug 24 06:51:35.408595 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application token = self._validate_token(token_id) Aug 24 06:51:35.408595 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:51:35.408595 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:51:35.408595 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:51:35.408595 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:51:35.408595 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:51:35.408595 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 24 06:51:35.408595 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:51:35.408595 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:51:35.408595 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:51:35.408595 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:51:35.409978 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:51:35.409978 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:51:35.409978 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:51:35.409978 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:51:35.409978 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 179, in _validate_token Aug 24 06:51:35.409978 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application token.mint(token_id, issued_at) Aug 24 06:51:35.409978 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 579, in mint Aug 24 06:51:35.409978 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application self._validate_token_resources() Aug 24 06:51:35.409978 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 464, in _validate_token_resources Aug 24 06:51:35.409978 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application if self.project and not self.project.get('enabled'): Aug 24 06:51:35.409978 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 162, in project Aug 24 06:51:35.409978 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application self.__project = PROVIDERS.resource_api.get_project( Aug 24 06:51:35.409978 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:51:35.409978 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:51:35.409978 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:51:35.409978 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:51:35.409978 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:51:35.411608 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:51:35.411608 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:51:35.411608 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application with Lock( Aug 24 06:51:35.411608 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:51:35.411608 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:51:35.411608 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:51:35.411608 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:51:35.411608 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:51:35.411608 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:51:35.411608 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:51:35.411608 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:51:35.411608 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 06:51:35.411608 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 06:51:35.411608 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 06:51:35.411608 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 06:51:35.411608 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 06:51:35.411608 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 06:51:35.413271 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: df659b3447644e7dbe0f124b881e4344. Aug 24 06:51:35.413271 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: ERROR keystone.server.flask.application Aug 24 06:51:35.413271 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1217/2435] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:51:35 2022] GET /identity/v3/auth/tokens => generated 113 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Aug 24 06:51:36.969792 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef2920c-15dc-476a-b7cb-89622e8a4530 service cinder] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:36.971260 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-9ef2920c-15dc-476a-b7cb-89622e8a4530 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:51:36.982113 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ae867d-06e9-472b-af42-954629dc7b45 service glance] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:36.984346 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-e4ae867d-06e9-472b-af42-954629dc7b45 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:51:36.997979 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ae867d-06e9-472b-af42-954629dc7b45 service glance] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:37.002750 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4ab0b4-73f4-4445-bf66-993a8557de76 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c4def55ac0e48b29836bc5c64bc7402', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:37.003366 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4ab0b4-73f4-4445-bf66-993a8557de76 service cinder] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:37.003663 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4ab0b4-73f4-4445-bf66-993a8557de76 service cinder] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:37.003968 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4ab0b4-73f4-4445-bf66-993a8557de76 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:37.045080 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef2920c-15dc-476a-b7cb-89622e8a4530 service cinder] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:37.049066 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe4f9b5-1e6b-4a7e-a07e-1c7ee9fccb46 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bb6c79db75e4d169331c02133d1ec32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:37.049728 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe4f9b5-1e6b-4a7e-a07e-1c7ee9fccb46 service neutron] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:37.050136 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe4f9b5-1e6b-4a7e-a07e-1c7ee9fccb46 service neutron] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:37.050309 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe4f9b5-1e6b-4a7e-a07e-1c7ee9fccb46 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:37.052100 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-7fe4f9b5-1e6b-4a7e-a07e-1c7ee9fccb46 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:51:37.072008 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4ab0b4-73f4-4445-bf66-993a8557de76 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:37.073970 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4ab0b4-73f4-4445-bf66-993a8557de76 service cinder] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:37.084851 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1218/2436] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:51:36 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 06:51:37.147023 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe4f9b5-1e6b-4a7e-a07e-1c7ee9fccb46 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:37.148477 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe4f9b5-1e6b-4a7e-a07e-1c7ee9fccb46 service neutron] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:37.170639 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1219/2437] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:51:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:51:37.980293 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4ab0b4-73f4-4445-bf66-993a8557de76 service cinder] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:37.981774 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-8a4ab0b4-73f4-4445-bf66-993a8557de76 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:51:37.996246 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4ab0b4-73f4-4445-bf66-993a8557de76 service cinder] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:38.004618 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a04d10f-6b96-4583-997e-07d220fb094a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:38.005283 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a04d10f-6b96-4583-997e-07d220fb094a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:38.005625 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a04d10f-6b96-4583-997e-07d220fb094a admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:38.005982 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a04d10f-6b96-4583-997e-07d220fb094a admin admin] PATH_INFO: `/v3/users/4cb18bdb6d404cd5b303ce0a29212e4f` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:38.007469 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a04d10f-6b96-4583-997e-07d220fb094a admin admin] RBAC: Authorizing `identity:delete_user(user_id=4cb18bdb6d404cd5b303ce0a29212e4f)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:38.009588 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a04d10f-6b96-4583-997e-07d220fb094a admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:38.293456 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-1a04d10f-6b96-4583-997e-07d220fb094a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4cb18bdb6d404cd5b303ce0a29212e4f'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:38.302243 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-1a04d10f-6b96-4583-997e-07d220fb094a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4cb18bdb6d404cd5b303ce0a29212e4f'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:38.302998 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-1a04d10f-6b96-4583-997e-07d220fb094a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4cb18bdb6d404cd5b303ce0a29212e4f'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:38.378134 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-1a04d10f-6b96-4583-997e-07d220fb094a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4cb18bdb6d404cd5b303ce0a29212e4f'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:38.395678 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-1a04d10f-6b96-4583-997e-07d220fb094a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4cb18bdb6d404cd5b303ce0a29212e4f'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:38.397481 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:51:38.404713 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1219/2438] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Aug 24 06:51:37 2022] DELETE /identity/v3/users/4cb18bdb6d404cd5b303ce0a29212e4f => generated 0 bytes in 423 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:51:38.701240 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe4f9b5-1e6b-4a7e-a07e-1c7ee9fccb46 service neutron] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:38.702971 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-7fe4f9b5-1e6b-4a7e-a07e-1c7ee9fccb46 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:51:38.723061 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe4f9b5-1e6b-4a7e-a07e-1c7ee9fccb46 service neutron] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:38.746151 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:51:38.783059 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3194dfbf-38ac-4b62-86a4-14316e0bee9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:38.783683 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3194dfbf-38ac-4b62-86a4-14316e0bee9d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:38.783968 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3194dfbf-38ac-4b62-86a4-14316e0bee9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:38.784242 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3194dfbf-38ac-4b62-86a4-14316e0bee9d admin admin] PATH_INFO: `/v3/users/f7092d51b8234bf0aced8af4034b759b` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:38.785669 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3194dfbf-38ac-4b62-86a4-14316e0bee9d admin admin] RBAC: Authorizing `identity:delete_user(user_id=f7092d51b8234bf0aced8af4034b759b)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:38.787622 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3194dfbf-38ac-4b62-86a4-14316e0bee9d admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:38.799050 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a04d10f-6b96-4583-997e-07d220fb094a admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:38.799050 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-1a04d10f-6b96-4583-997e-07d220fb094a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:51:38.816517 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a04d10f-6b96-4583-997e-07d220fb094a admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:38.821990 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36f6d2a-4380-46fc-9489-8e08124a94b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:38.822730 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36f6d2a-4380-46fc-9489-8e08124a94b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:38.823111 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36f6d2a-4380-46fc-9489-8e08124a94b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:38.823428 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36f6d2a-4380-46fc-9489-8e08124a94b5 admin admin] PATH_INFO: `/v3/projects/7315c7714a7f40ed852dde74d497d231` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:38.824638 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36f6d2a-4380-46fc-9489-8e08124a94b5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7315c7714a7f40ed852dde74d497d231)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:38.826432 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36f6d2a-4380-46fc-9489-8e08124a94b5 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:39.021176 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-3194dfbf-38ac-4b62-86a4-14316e0bee9d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f7092d51b8234bf0aced8af4034b759b'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:39.021553 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-b36f6d2a-4380-46fc-9489-8e08124a94b5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7315c7714a7f40ed852dde74d497d231', 'request_id': 'req-b36f6d2a-4380-46fc-9489-8e08124a94b5'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:39.029224 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-3194dfbf-38ac-4b62-86a4-14316e0bee9d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f7092d51b8234bf0aced8af4034b759b'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:39.029663 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-b36f6d2a-4380-46fc-9489-8e08124a94b5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7315c7714a7f40ed852dde74d497d231', 'request_id': 'req-b36f6d2a-4380-46fc-9489-8e08124a94b5'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:39.029947 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-3194dfbf-38ac-4b62-86a4-14316e0bee9d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f7092d51b8234bf0aced8af4034b759b'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:39.030693 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-3194dfbf-38ac-4b62-86a4-14316e0bee9d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f7092d51b8234bf0aced8af4034b759b'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:39.037431 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-b36f6d2a-4380-46fc-9489-8e08124a94b5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7315c7714a7f40ed852dde74d497d231', 'request_id': 'req-b36f6d2a-4380-46fc-9489-8e08124a94b5'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:39.120485 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-3194dfbf-38ac-4b62-86a4-14316e0bee9d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f7092d51b8234bf0aced8af4034b759b'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:39.139531 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1220/2439] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Aug 24 06:51:38 2022] DELETE /identity/v3/users/f7092d51b8234bf0aced8af4034b759b => generated 0 bytes in 439 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:51:39.144349 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a1dbd9-73b8-4210-a4c8-831509c2ace1 None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:39.144472 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a1dbd9-73b8-4210-a4c8-831509c2ace1 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:39.144756 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a1dbd9-73b8-4210-a4c8-831509c2ace1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:39.164799 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-c1a1dbd9-73b8-4210-a4c8-831509c2ace1 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:51:39.165995 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-c1a1dbd9-73b8-4210-a4c8-831509c2ace1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:51:39.211006 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-b36f6d2a-4380-46fc-9489-8e08124a94b5 admin admin] The token cache is being invalidate because project 7315c7714a7f40ed852dde74d497d231 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81852) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:51:39.211325 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-b36f6d2a-4380-46fc-9489-8e08124a94b5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:39.211978 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-b36f6d2a-4380-46fc-9489-8e08124a94b5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:39.214330 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1220/2440] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Aug 24 06:51:38 2022] DELETE /identity/v3/projects/7315c7714a7f40ed852dde74d497d231 => generated 0 bytes in 421 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 06:51:39.220050 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9de158-6b48-4608-b7d8-81ad8f16d94a None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:39.225891 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9de158-6b48-4608-b7d8-81ad8f16d94a None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:39.225891 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9de158-6b48-4608-b7d8-81ad8f16d94a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:39.226437 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1221/2441] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:51:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 06:51:39.236678 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a1dbd9-73b8-4210-a4c8-831509c2ace1 None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:39.238132 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-c1a1dbd9-73b8-4210-a4c8-831509c2ace1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:51:39.241368 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-eb9de158-6b48-4608-b7d8-81ad8f16d94a None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:51:39.242566 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-eb9de158-6b48-4608-b7d8-81ad8f16d94a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:51:39.252922 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1221/2442] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:51:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:51:39.260990 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53bd53e-4897-4398-9448-62ecb4c3860f None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:39.260990 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53bd53e-4897-4398-9448-62ecb4c3860f None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:39.261302 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53bd53e-4897-4398-9448-62ecb4c3860f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:39.281006 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-a53bd53e-4897-4398-9448-62ecb4c3860f None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:51:39.282216 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-a53bd53e-4897-4398-9448-62ecb4c3860f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:51:39.290544 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1222/2443] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:51:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 06:51:39.292189 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:51:39.295423 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53bd53e-4897-4398-9448-62ecb4c3860f None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:39.303993 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a1dbd9-73b8-4210-a4c8-831509c2ace1 None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:39.304875 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53bd53e-4897-4398-9448-62ecb4c3860f None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:39.307883 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abe1d1a-72c3-40c0-9d2d-9e853576387b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bb6c79db75e4d169331c02133d1ec32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:39.308518 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abe1d1a-72c3-40c0-9d2d-9e853576387b service neutron] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:39.308800 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abe1d1a-72c3-40c0-9d2d-9e853576387b service neutron] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:39.309074 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abe1d1a-72c3-40c0-9d2d-9e853576387b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:39.310408 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23ecae0-e0d2-4a4b-befd-01bddbb91458 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:39.311021 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23ecae0-e0d2-4a4b-befd-01bddbb91458 admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:39.311302 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23ecae0-e0d2-4a4b-befd-01bddbb91458 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:39.311574 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23ecae0-e0d2-4a4b-befd-01bddbb91458 admin admin] PATH_INFO: `/v3/domains` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:39.312230 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23ecae0-e0d2-4a4b-befd-01bddbb91458 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:51:39.312631 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23ecae0-e0d2-4a4b-befd-01bddbb91458 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:39.314258 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23ecae0-e0d2-4a4b-befd-01bddbb91458 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:39.318861 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7abe1d1a-72c3-40c0-9d2d-9e853576387b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:39.320334 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7abe1d1a-72c3-40c0-9d2d-9e853576387b service neutron] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:39.326408 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1223/2444] 10.222.0.62 () {58 vars in 1153 bytes} [Wed Aug 24 06:51:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 06:51:39.330790 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1222/2445] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:51:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 06:51:39.330972 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23ecae0-e0d2-4a4b-befd-01bddbb91458 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:39.341595 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23ecae0-e0d2-4a4b-befd-01bddbb91458 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:39.347618 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66133dc-c90b-44f1-8a79-421aa5d09398 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:39.348380 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66133dc-c90b-44f1-8a79-421aa5d09398 admin admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:39.349073 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66133dc-c90b-44f1-8a79-421aa5d09398 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:39.349595 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66133dc-c90b-44f1-8a79-421aa5d09398 admin admin] PATH_INFO: `/v3/projects` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:39.351060 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66133dc-c90b-44f1-8a79-421aa5d09398 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:39.353892 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66133dc-c90b-44f1-8a79-421aa5d09398 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:39.456875 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1224/2446] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Aug 24 06:51:39 2022] POST /identity/v3/projects => generated 389 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 06:51:39.461895 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abe1d1a-72c3-40c0-9d2d-9e853576387b service neutron] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:39.471863 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abe1d1a-72c3-40c0-9d2d-9e853576387b service neutron] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:39.529101 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0939619-9c6a-411a-ab5e-e42501d048bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:39.529623 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0939619-9c6a-411a-ab5e-e42501d048bd admin admin] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:39.529905 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0939619-9c6a-411a-ab5e-e42501d048bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:39.530178 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0939619-9c6a-411a-ab5e-e42501d048bd admin admin] PATH_INFO: `/v3/users` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:39.530812 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0939619-9c6a-411a-ab5e-e42501d048bd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:39.532809 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0939619-9c6a-411a-ab5e-e42501d048bd admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:39.712236 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1223/2447] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Aug 24 06:51:39 2022] POST /identity/v3/users => generated 295 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:51:39.717369 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66133dc-c90b-44f1-8a79-421aa5d09398 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:39.732086 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66133dc-c90b-44f1-8a79-421aa5d09398 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:39.738126 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3baf31c-2d12-4c7d-ba51-1d33bc9d025b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:39.739122 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3baf31c-2d12-4c7d-ba51-1d33bc9d025b admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:39.739649 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3baf31c-2d12-4c7d-ba51-1d33bc9d025b admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:39.740437 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3baf31c-2d12-4c7d-ba51-1d33bc9d025b admin admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:39.742316 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3baf31c-2d12-4c7d-ba51-1d33bc9d025b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:39.743373 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3baf31c-2d12-4c7d-ba51-1d33bc9d025b admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:39.760925 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1225/2448] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Aug 24 06:51:39 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:51:39.765492 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0939619-9c6a-411a-ab5e-e42501d048bd admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:39.776376 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0939619-9c6a-411a-ab5e-e42501d048bd admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:39.781961 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b89712-3767-4e65-85e9-62a7839de268 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:39.786721 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b89712-3767-4e65-85e9-62a7839de268 admin admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:39.786721 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b89712-3767-4e65-85e9-62a7839de268 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:39.786721 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b89712-3767-4e65-85e9-62a7839de268 admin admin] PATH_INFO: `/v3/roles` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:39.786721 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b89712-3767-4e65-85e9-62a7839de268 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:39.786721 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b89712-3767-4e65-85e9-62a7839de268 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:39.797097 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1224/2449] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Aug 24 06:51:39 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 06:51:39.801062 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3baf31c-2d12-4c7d-ba51-1d33bc9d025b admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:39.811562 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3baf31c-2d12-4c7d-ba51-1d33bc9d025b admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:39.816208 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5c371f-491c-4d22-add9-197a0499cbd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:39.817931 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5c371f-491c-4d22-add9-197a0499cbd9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:39.817931 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5c371f-491c-4d22-add9-197a0499cbd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:39.817931 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5c371f-491c-4d22-add9-197a0499cbd9 admin admin] PATH_INFO: `/v3/projects/d2aa8824715e47ab9ef1d8b459050c7f/users/cbc0eae15e4044c498fdd8412e0f038d/roles/580aa7dbb38d4cff8f9d1bb8983f99ef` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:39.833215 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5c371f-491c-4d22-add9-197a0499cbd9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d2aa8824715e47ab9ef1d8b459050c7f, user_id=cbc0eae15e4044c498fdd8412e0f038d, role_id=580aa7dbb38d4cff8f9d1bb8983f99ef)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:39.835909 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5c371f-491c-4d22-add9-197a0499cbd9 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:39.845752 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b89712-3767-4e65-85e9-62a7839de268 admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:39.849325 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-f0b89712-3767-4e65-85e9-62a7839de268 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:51:39.862358 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b89712-3767-4e65-85e9-62a7839de268 admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:39.867327 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26350fc6-50eb-45ca-820a-3ad141e48e5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:39.868024 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26350fc6-50eb-45ca-820a-3ad141e48e5d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:39.868903 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26350fc6-50eb-45ca-820a-3ad141e48e5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:39.869211 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26350fc6-50eb-45ca-820a-3ad141e48e5d admin admin] PATH_INFO: `/v3/projects/4a506cfdbf2b45e4abcd29b4fdec4625` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:39.870563 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26350fc6-50eb-45ca-820a-3ad141e48e5d admin admin] RBAC: Authorizing `identity:delete_project(project_id=4a506cfdbf2b45e4abcd29b4fdec4625)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:39.872565 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26350fc6-50eb-45ca-820a-3ad141e48e5d admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:39.932269 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1226/2450] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Aug 24 06:51:39 2022] PUT /identity/v3/projects/d2aa8824715e47ab9ef1d8b459050c7f/users/cbc0eae15e4044c498fdd8412e0f038d/roles/580aa7dbb38d4cff8f9d1bb8983f99ef => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:51:39.944336 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0890041-029e-4bdc-b881-d58d28ab575f None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:39.944647 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0890041-029e-4bdc-b881-d58d28ab575f None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:39.944917 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0890041-029e-4bdc-b881-d58d28ab575f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:39.975688 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-d0890041-029e-4bdc-b881-d58d28ab575f None None] MFA Rules not processed for user `cbc0eae15e4044c498fdd8412e0f038d`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:51:39.976992 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-d0890041-029e-4bdc-b881-d58d28ab575f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:51:40.022100 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:51:40.025444 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-26350fc6-50eb-45ca-820a-3ad141e48e5d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4a506cfdbf2b45e4abcd29b4fdec4625', 'request_id': 'req-26350fc6-50eb-45ca-820a-3ad141e48e5d'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:40.031760 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-26350fc6-50eb-45ca-820a-3ad141e48e5d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4a506cfdbf2b45e4abcd29b4fdec4625', 'request_id': 'req-26350fc6-50eb-45ca-820a-3ad141e48e5d'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:40.040461 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-26350fc6-50eb-45ca-820a-3ad141e48e5d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4a506cfdbf2b45e4abcd29b4fdec4625', 'request_id': 'req-26350fc6-50eb-45ca-820a-3ad141e48e5d'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:40.045739 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1227/2451] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:51:39 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 06:51:40.056380 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0890041-029e-4bdc-b881-d58d28ab575f None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:40.057751 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-d0890041-029e-4bdc-b881-d58d28ab575f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:51:40.115577 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0890041-029e-4bdc-b881-d58d28ab575f None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:40.133745 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-26350fc6-50eb-45ca-820a-3ad141e48e5d admin admin] The token cache is being invalidate because project 4a506cfdbf2b45e4abcd29b4fdec4625 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81851) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:51:40.134037 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-26350fc6-50eb-45ca-820a-3ad141e48e5d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:40.135067 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-26350fc6-50eb-45ca-820a-3ad141e48e5d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:40.136321 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1225/2452] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Aug 24 06:51:39 2022] DELETE /identity/v3/projects/4a506cfdbf2b45e4abcd29b4fdec4625 => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 06:51:40.140390 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26350fc6-50eb-45ca-820a-3ad141e48e5d admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:40.141759 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-26350fc6-50eb-45ca-820a-3ad141e48e5d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:51:40.166240 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead15c78-51d3-4fd7-8a30-60f055afd17a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c4def55ac0e48b29836bc5c64bc7402', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:40.166866 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead15c78-51d3-4fd7-8a30-60f055afd17a service cinder] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:40.167140 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead15c78-51d3-4fd7-8a30-60f055afd17a service cinder] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:40.167417 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead15c78-51d3-4fd7-8a30-60f055afd17a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:40.169804 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-ead15c78-51d3-4fd7-8a30-60f055afd17a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:51:40.193937 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:51:40.209182 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26350fc6-50eb-45ca-820a-3ad141e48e5d admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:40.214521 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace64f83-7e86-408c-8e1a-800e2d0e5ac5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:40.215156 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace64f83-7e86-408c-8e1a-800e2d0e5ac5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:40.216069 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace64f83-7e86-408c-8e1a-800e2d0e5ac5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:40.216378 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace64f83-7e86-408c-8e1a-800e2d0e5ac5 admin admin] PATH_INFO: `/v3/users/f1e144328fdd4ddcbf583e2a41619963` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:40.217529 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace64f83-7e86-408c-8e1a-800e2d0e5ac5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f1e144328fdd4ddcbf583e2a41619963)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:40.219458 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace64f83-7e86-408c-8e1a-800e2d0e5ac5 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:40.233949 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead15c78-51d3-4fd7-8a30-60f055afd17a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:40.235417 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead15c78-51d3-4fd7-8a30-60f055afd17a service cinder] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:40.244849 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1228/2453] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:51:40 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 06:51:40.474547 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-ace64f83-7e86-408c-8e1a-800e2d0e5ac5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f1e144328fdd4ddcbf583e2a41619963'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:40.481720 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-ace64f83-7e86-408c-8e1a-800e2d0e5ac5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f1e144328fdd4ddcbf583e2a41619963'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:40.482411 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-ace64f83-7e86-408c-8e1a-800e2d0e5ac5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f1e144328fdd4ddcbf583e2a41619963'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:40.483012 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-ace64f83-7e86-408c-8e1a-800e2d0e5ac5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f1e144328fdd4ddcbf583e2a41619963'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:40.580424 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-ace64f83-7e86-408c-8e1a-800e2d0e5ac5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f1e144328fdd4ddcbf583e2a41619963'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:40.599378 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1226/2454] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Aug 24 06:51:40 2022] DELETE /identity/v3/users/f1e144328fdd4ddcbf583e2a41619963 => generated 0 bytes in 459 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 06:51:41.096884 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead15c78-51d3-4fd7-8a30-60f055afd17a service cinder] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:41.123454 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead15c78-51d3-4fd7-8a30-60f055afd17a service cinder] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:41.204537 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=100 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:51:41.210749 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a9ab87-2171-481a-aed8-3c801d444a1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:41.211461 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a9ab87-2171-481a-aed8-3c801d444a1f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:41.211803 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a9ab87-2171-481a-aed8-3c801d444a1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:41.212141 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a9ab87-2171-481a-aed8-3c801d444a1f admin admin] PATH_INFO: `/v3/projects/7c468f9d6a6f43b0885b1d76feb1add9` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:41.213463 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a9ab87-2171-481a-aed8-3c801d444a1f admin admin] RBAC: Authorizing `identity:delete_project(project_id=7c468f9d6a6f43b0885b1d76feb1add9)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:41.215654 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a9ab87-2171-481a-aed8-3c801d444a1f admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:41.349531 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-28a9ab87-2171-481a-aed8-3c801d444a1f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7c468f9d6a6f43b0885b1d76feb1add9', 'request_id': 'req-28a9ab87-2171-481a-aed8-3c801d444a1f'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:41.357697 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-28a9ab87-2171-481a-aed8-3c801d444a1f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7c468f9d6a6f43b0885b1d76feb1add9', 'request_id': 'req-28a9ab87-2171-481a-aed8-3c801d444a1f'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:41.364287 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-28a9ab87-2171-481a-aed8-3c801d444a1f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7c468f9d6a6f43b0885b1d76feb1add9', 'request_id': 'req-28a9ab87-2171-481a-aed8-3c801d444a1f'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:41.459933 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-28a9ab87-2171-481a-aed8-3c801d444a1f admin admin] The token cache is being invalidate because project 7c468f9d6a6f43b0885b1d76feb1add9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81852) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:51:41.460227 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-28a9ab87-2171-481a-aed8-3c801d444a1f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:41.460988 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-28a9ab87-2171-481a-aed8-3c801d444a1f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:41.462676 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1229/2455] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Aug 24 06:51:41 2022] DELETE /identity/v3/projects/7c468f9d6a6f43b0885b1d76feb1add9 => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 06:51:41.468549 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249f121d-5bfc-4b8c-9320-e72ecc2cd1c4 None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:41.469042 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249f121d-5bfc-4b8c-9320-e72ecc2cd1c4 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:41.469160 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249f121d-5bfc-4b8c-9320-e72ecc2cd1c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:41.534025 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-249f121d-5bfc-4b8c-9320-e72ecc2cd1c4 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:51:41.535263 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-249f121d-5bfc-4b8c-9320-e72ecc2cd1c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:51:41.597101 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1227/2456] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:51:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:51:41.604957 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d34d71-b85a-4e27-a65d-4e6e68a6e705 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:41.605179 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d34d71-b85a-4e27-a65d-4e6e68a6e705 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:41.605449 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d34d71-b85a-4e27-a65d-4e6e68a6e705 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:41.625665 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-05d34d71-b85a-4e27-a65d-4e6e68a6e705 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:51:41.626860 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-05d34d71-b85a-4e27-a65d-4e6e68a6e705 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:51:41.634774 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1230/2457] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:51:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:51:41.639250 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249f121d-5bfc-4b8c-9320-e72ecc2cd1c4 None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:41.649717 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249f121d-5bfc-4b8c-9320-e72ecc2cd1c4 None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:41.655783 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73804b58-ecbd-4ed6-92d4-e02a3c2daf99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:41.656402 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73804b58-ecbd-4ed6-92d4-e02a3c2daf99 admin admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:41.656712 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73804b58-ecbd-4ed6-92d4-e02a3c2daf99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:41.656987 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73804b58-ecbd-4ed6-92d4-e02a3c2daf99 admin admin] PATH_INFO: `/v3/domains` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:41.657639 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73804b58-ecbd-4ed6-92d4-e02a3c2daf99 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:51:41.657967 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73804b58-ecbd-4ed6-92d4-e02a3c2daf99 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:41.659930 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73804b58-ecbd-4ed6-92d4-e02a3c2daf99 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:41.673118 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1228/2458] 10.222.0.62 () {58 vars in 1153 bytes} [Wed Aug 24 06:51:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:51:41.677543 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d34d71-b85a-4e27-a65d-4e6e68a6e705 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:41.687851 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d34d71-b85a-4e27-a65d-4e6e68a6e705 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:41.692588 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdcdd5e-e64a-42b4-b8c5-f71cce9f0a57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:41.693202 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdcdd5e-e64a-42b4-b8c5-f71cce9f0a57 admin admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:41.694275 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdcdd5e-e64a-42b4-b8c5-f71cce9f0a57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:41.694275 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdcdd5e-e64a-42b4-b8c5-f71cce9f0a57 admin admin] PATH_INFO: `/v3/projects` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:41.694507 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cdcdd5e-e64a-42b4-b8c5-f71cce9f0a57 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:41.696261 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cdcdd5e-e64a-42b4-b8c5-f71cce9f0a57 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:41.866078 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1231/2459] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Aug 24 06:51:41 2022] POST /identity/v3/projects => generated 383 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:51:41.870708 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73804b58-ecbd-4ed6-92d4-e02a3c2daf99 admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:41.881514 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73804b58-ecbd-4ed6-92d4-e02a3c2daf99 admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:41.903912 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:51:41.941078 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816e5c28-a932-4d88-a1df-895dd88274c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:41.941678 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816e5c28-a932-4d88-a1df-895dd88274c4 admin admin] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:41.941954 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816e5c28-a932-4d88-a1df-895dd88274c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:41.942251 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816e5c28-a932-4d88-a1df-895dd88274c4 admin admin] PATH_INFO: `/v3/users` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:41.942909 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816e5c28-a932-4d88-a1df-895dd88274c4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:41.944920 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816e5c28-a932-4d88-a1df-895dd88274c4 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:42.100808 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1229/2460] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Aug 24 06:51:41 2022] POST /identity/v3/users => generated 292 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 06:51:42.105241 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdcdd5e-e64a-42b4-b8c5-f71cce9f0a57 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:42.121972 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdcdd5e-e64a-42b4-b8c5-f71cce9f0a57 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:42.123035 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae459ca6-0c2c-4c3b-aa5b-103e2a0f6775 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:42.123721 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae459ca6-0c2c-4c3b-aa5b-103e2a0f6775 admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:42.124066 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae459ca6-0c2c-4c3b-aa5b-103e2a0f6775 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:42.124403 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae459ca6-0c2c-4c3b-aa5b-103e2a0f6775 admin admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:42.125166 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae459ca6-0c2c-4c3b-aa5b-103e2a0f6775 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:42.126877 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae459ca6-0c2c-4c3b-aa5b-103e2a0f6775 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:42.141116 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1232/2461] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Aug 24 06:51:42 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:51:42.142510 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816e5c28-a932-4d88-a1df-895dd88274c4 admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:42.156558 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816e5c28-a932-4d88-a1df-895dd88274c4 admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:42.161533 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e439c6-f75f-4d01-91ca-83fefa209d69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:42.162153 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e439c6-f75f-4d01-91ca-83fefa209d69 admin admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:42.162473 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e439c6-f75f-4d01-91ca-83fefa209d69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:42.162755 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e439c6-f75f-4d01-91ca-83fefa209d69 admin admin] PATH_INFO: `/v3/roles` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:42.163415 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e439c6-f75f-4d01-91ca-83fefa209d69 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:42.165105 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e439c6-f75f-4d01-91ca-83fefa209d69 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:42.175930 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1230/2462] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Aug 24 06:51:42 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:51:42.180569 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae459ca6-0c2c-4c3b-aa5b-103e2a0f6775 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:42.191324 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae459ca6-0c2c-4c3b-aa5b-103e2a0f6775 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:42.195876 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b60df80-42f7-4d00-8d70-03b5960fdde6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:42.196551 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b60df80-42f7-4d00-8d70-03b5960fdde6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:42.196832 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b60df80-42f7-4d00-8d70-03b5960fdde6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:42.197106 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b60df80-42f7-4d00-8d70-03b5960fdde6 admin admin] PATH_INFO: `/v3/projects/4f70d27dc57741c491584a94951c1d02/users/29e210dd6d5849d2918dde5e0e8328c5/roles/580aa7dbb38d4cff8f9d1bb8983f99ef` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:42.213598 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b60df80-42f7-4d00-8d70-03b5960fdde6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f70d27dc57741c491584a94951c1d02, user_id=29e210dd6d5849d2918dde5e0e8328c5, role_id=580aa7dbb38d4cff8f9d1bb8983f99ef)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:42.216385 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b60df80-42f7-4d00-8d70-03b5960fdde6 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:42.347405 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1233/2463] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Aug 24 06:51:42 2022] PUT /identity/v3/projects/4f70d27dc57741c491584a94951c1d02/users/29e210dd6d5849d2918dde5e0e8328c5/roles/580aa7dbb38d4cff8f9d1bb8983f99ef => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:51:42.360247 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b3c38f-9268-4e70-9537-53037c61ed28 None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:42.360593 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b3c38f-9268-4e70-9537-53037c61ed28 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:42.360886 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b3c38f-9268-4e70-9537-53037c61ed28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:42.392835 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-b4b3c38f-9268-4e70-9537-53037c61ed28 None None] MFA Rules not processed for user `29e210dd6d5849d2918dde5e0e8328c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:51:42.394150 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-b4b3c38f-9268-4e70-9537-53037c61ed28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:51:42.464660 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1231/2464] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:51:42 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:51:42.469968 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b60df80-42f7-4d00-8d70-03b5960fdde6 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:42.533482 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b3c38f-9268-4e70-9537-53037c61ed28 None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:42.538568 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-b4b3c38f-9268-4e70-9537-53037c61ed28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:51:42.539459 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b60df80-42f7-4d00-8d70-03b5960fdde6 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:42.545175 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d77f75d-77c9-400a-9c9a-9c53a5481ae3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:42.545790 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d77f75d-77c9-400a-9c9a-9c53a5481ae3 admin admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:42.546514 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d77f75d-77c9-400a-9c9a-9c53a5481ae3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:42.546794 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d77f75d-77c9-400a-9c9a-9c53a5481ae3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:42.547597 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d77f75d-77c9-400a-9c9a-9c53a5481ae3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:42.549679 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d77f75d-77c9-400a-9c9a-9c53a5481ae3 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:42.594861 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b3c38f-9268-4e70-9537-53037c61ed28 None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:42.598621 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38466683-740a-4ccf-883f-73655ee494c5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c4def55ac0e48b29836bc5c64bc7402', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:42.599221 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38466683-740a-4ccf-883f-73655ee494c5 service cinder] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:42.599493 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38466683-740a-4ccf-883f-73655ee494c5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:42.599759 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38466683-740a-4ccf-883f-73655ee494c5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:42.601521 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-38466683-740a-4ccf-883f-73655ee494c5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:51:42.645439 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:51:42.650028 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:51:42.655853 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1234/2465] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Aug 24 06:51:42 2022] POST /identity/v3/projects => generated 383 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:51:42.662760 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d77f75d-77c9-400a-9c9a-9c53a5481ae3 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:42.666291 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38466683-740a-4ccf-883f-73655ee494c5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:42.667914 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38466683-740a-4ccf-883f-73655ee494c5 service cinder] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:42.678922 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1232/2466] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:51:42 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 06:51:42.734885 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d77f75d-77c9-400a-9c9a-9c53a5481ae3 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:42.740723 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064a9a3e-918b-4737-93f9-fd880545ffe8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:42.741570 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064a9a3e-918b-4737-93f9-fd880545ffe8 admin admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:42.741979 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064a9a3e-918b-4737-93f9-fd880545ffe8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:42.742417 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064a9a3e-918b-4737-93f9-fd880545ffe8 admin admin] PATH_INFO: `/v3/users` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:42.743110 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064a9a3e-918b-4737-93f9-fd880545ffe8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:42.745358 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064a9a3e-918b-4737-93f9-fd880545ffe8 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:42.874825 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1235/2467] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Aug 24 06:51:42 2022] POST /identity/v3/users => generated 298 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:51:42.878871 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38466683-740a-4ccf-883f-73655ee494c5 service cinder] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:42.890488 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38466683-740a-4ccf-883f-73655ee494c5 service cinder] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:42.895760 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b21851d-4d3c-4748-8780-59314fe0646f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:42.896849 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b21851d-4d3c-4748-8780-59314fe0646f admin admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:42.897192 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b21851d-4d3c-4748-8780-59314fe0646f admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:42.897500 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b21851d-4d3c-4748-8780-59314fe0646f admin admin] PATH_INFO: `/v3/roles` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:42.898176 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b21851d-4d3c-4748-8780-59314fe0646f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:42.900630 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b21851d-4d3c-4748-8780-59314fe0646f admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:42.918965 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1233/2468] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Aug 24 06:51:42 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:51:42.920652 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064a9a3e-918b-4737-93f9-fd880545ffe8 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:42.930639 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064a9a3e-918b-4737-93f9-fd880545ffe8 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:42.935515 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6befb854-6f37-4f2e-a488-efdbabdcddad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:42.936164 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6befb854-6f37-4f2e-a488-efdbabdcddad admin admin] REQUEST_METHOD: `PUT` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:42.936534 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6befb854-6f37-4f2e-a488-efdbabdcddad admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:42.942796 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6befb854-6f37-4f2e-a488-efdbabdcddad admin admin] PATH_INFO: `/v3/projects/6b0f57e7b36d418f9cd961ccae41ad01/users/8211405d9e7449fb88a2953c0b4d6bd6/roles/49e47cd157d743f6905f1db07ec878a7` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:42.953251 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6befb854-6f37-4f2e-a488-efdbabdcddad admin admin] RBAC: Authorizing `identity:create_grant(project_id=6b0f57e7b36d418f9cd961ccae41ad01, user_id=8211405d9e7449fb88a2953c0b4d6bd6, role_id=49e47cd157d743f6905f1db07ec878a7)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:42.955948 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6befb854-6f37-4f2e-a488-efdbabdcddad admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:43.098209 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1236/2469] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Aug 24 06:51:42 2022] PUT /identity/v3/projects/6b0f57e7b36d418f9cd961ccae41ad01/users/8211405d9e7449fb88a2953c0b4d6bd6/roles/49e47cd157d743f6905f1db07ec878a7 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:51:43.102664 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b21851d-4d3c-4748-8780-59314fe0646f admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:43.113341 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b21851d-4d3c-4748-8780-59314fe0646f admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:43.184846 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156f6bef-6d0c-4295-a203-b8c1319f2361 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:43.184846 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156f6bef-6d0c-4295-a203-b8c1319f2361 admin admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:43.184846 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156f6bef-6d0c-4295-a203-b8c1319f2361 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:43.184846 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156f6bef-6d0c-4295-a203-b8c1319f2361 admin admin] PATH_INFO: `/v3/roles` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:43.185652 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156f6bef-6d0c-4295-a203-b8c1319f2361 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:43.185652 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156f6bef-6d0c-4295-a203-b8c1319f2361 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:43.196004 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1234/2470] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Aug 24 06:51:43 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:51:43.200821 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6befb854-6f37-4f2e-a488-efdbabdcddad admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:43.213171 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6befb854-6f37-4f2e-a488-efdbabdcddad admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:43.217964 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae924a7-6add-4f71-9541-6426236990bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:43.218698 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae924a7-6add-4f71-9541-6426236990bd admin admin] REQUEST_METHOD: `PUT` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:43.218981 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae924a7-6add-4f71-9541-6426236990bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:43.219482 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae924a7-6add-4f71-9541-6426236990bd admin admin] PATH_INFO: `/v3/projects/6b0f57e7b36d418f9cd961ccae41ad01/users/8211405d9e7449fb88a2953c0b4d6bd6/roles/49e47cd157d743f6905f1db07ec878a7` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:43.221199 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae924a7-6add-4f71-9541-6426236990bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=6b0f57e7b36d418f9cd961ccae41ad01, user_id=8211405d9e7449fb88a2953c0b4d6bd6, role_id=49e47cd157d743f6905f1db07ec878a7)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:43.223958 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae924a7-6add-4f71-9541-6426236990bd admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:43.234842 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1237/2471] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Aug 24 06:51:43 2022] PUT /identity/v3/projects/6b0f57e7b36d418f9cd961ccae41ad01/users/8211405d9e7449fb88a2953c0b4d6bd6/roles/49e47cd157d743f6905f1db07ec878a7 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:51:43.245420 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce595399-bde8-4eea-9cc0-d3ba17959ae3 None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:43.245799 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce595399-bde8-4eea-9cc0-d3ba17959ae3 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:43.246081 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce595399-bde8-4eea-9cc0-d3ba17959ae3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:43.275798 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-ce595399-bde8-4eea-9cc0-d3ba17959ae3 None None] MFA Rules not processed for user `8211405d9e7449fb88a2953c0b4d6bd6`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:51:43.277075 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-ce595399-bde8-4eea-9cc0-d3ba17959ae3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:51:43.348782 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1235/2472] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:51:43 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:51:43.361080 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae924a7-6add-4f71-9541-6426236990bd admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:43.363029 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-fae924a7-6add-4f71-9541-6426236990bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:51:43.549273 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae924a7-6add-4f71-9541-6426236990bd admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:43.553159 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c0de1c-be34-4a27-8b3f-63b237ac45ec service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c4def55ac0e48b29836bc5c64bc7402', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:43.553775 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c0de1c-be34-4a27-8b3f-63b237ac45ec service cinder] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:43.554049 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c0de1c-be34-4a27-8b3f-63b237ac45ec service cinder] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:43.554332 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c0de1c-be34-4a27-8b3f-63b237ac45ec service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:43.564627 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c0de1c-be34-4a27-8b3f-63b237ac45ec service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:43.566041 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c0de1c-be34-4a27-8b3f-63b237ac45ec service cinder] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:43.576054 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1238/2473] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:51:43 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:51:43.677507 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce595399-bde8-4eea-9cc0-d3ba17959ae3 None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:43.687137 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce595399-bde8-4eea-9cc0-d3ba17959ae3 None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:43.690932 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599e94d5-9918-4dce-9648-bf67a5ba1633 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c4def55ac0e48b29836bc5c64bc7402', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:43.691543 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599e94d5-9918-4dce-9648-bf67a5ba1633 service cinder] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:43.691823 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599e94d5-9918-4dce-9648-bf67a5ba1633 service cinder] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:43.692095 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599e94d5-9918-4dce-9648-bf67a5ba1633 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:43.748861 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:51:43.751063 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599e94d5-9918-4dce-9648-bf67a5ba1633 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:43.752541 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599e94d5-9918-4dce-9648-bf67a5ba1633 service cinder] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:43.761779 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1236/2474] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:51:43 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:51:48.156370 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c0de1c-be34-4a27-8b3f-63b237ac45ec service cinder] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:48.212743 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:51:48.219494 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c0de1c-be34-4a27-8b3f-63b237ac45ec service cinder] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:48.224285 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b899ec-79f4-4196-af58-87b27e35bfb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:48.224951 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b899ec-79f4-4196-af58-87b27e35bfb4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:48.225227 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b899ec-79f4-4196-af58-87b27e35bfb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:48.225496 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b899ec-79f4-4196-af58-87b27e35bfb4 admin admin] PATH_INFO: `/v3/users/29e210dd6d5849d2918dde5e0e8328c5` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:48.226661 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b899ec-79f4-4196-af58-87b27e35bfb4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=29e210dd6d5849d2918dde5e0e8328c5)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:48.228629 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b899ec-79f4-4196-af58-87b27e35bfb4 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:48.465702 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-47b899ec-79f4-4196-af58-87b27e35bfb4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '29e210dd6d5849d2918dde5e0e8328c5'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:48.482481 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-47b899ec-79f4-4196-af58-87b27e35bfb4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '29e210dd6d5849d2918dde5e0e8328c5'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:48.482481 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-47b899ec-79f4-4196-af58-87b27e35bfb4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '29e210dd6d5849d2918dde5e0e8328c5'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:48.571202 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-47b899ec-79f4-4196-af58-87b27e35bfb4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '29e210dd6d5849d2918dde5e0e8328c5'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:48.584018 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-47b899ec-79f4-4196-af58-87b27e35bfb4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '29e210dd6d5849d2918dde5e0e8328c5'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:48.590560 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1239/2475] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Aug 24 06:51:48 2022] DELETE /identity/v3/users/29e210dd6d5849d2918dde5e0e8328c5 => generated 0 bytes in 435 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:51:48.595157 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cd2de9-006d-476a-bce4-373bbd3ea171 None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:48.595710 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cd2de9-006d-476a-bce4-373bbd3ea171 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:48.595710 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cd2de9-006d-476a-bce4-373bbd3ea171 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:48.615150 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-87cd2de9-006d-476a-bce4-373bbd3ea171 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:51:48.616478 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-87cd2de9-006d-476a-bce4-373bbd3ea171 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:51:48.678077 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1237/2476] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:51:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:51:48.688428 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b899ec-79f4-4196-af58-87b27e35bfb4 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:48.689861 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-47b899ec-79f4-4196-af58-87b27e35bfb4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:51:48.758875 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b899ec-79f4-4196-af58-87b27e35bfb4 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:48.764677 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21118e0f-62db-4eb4-9edd-25a33cb8a522 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bb6c79db75e4d169331c02133d1ec32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:48.765538 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21118e0f-62db-4eb4-9edd-25a33cb8a522 service neutron] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:48.765990 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21118e0f-62db-4eb4-9edd-25a33cb8a522 service neutron] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:48.766445 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21118e0f-62db-4eb4-9edd-25a33cb8a522 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:48.783121 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21118e0f-62db-4eb4-9edd-25a33cb8a522 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:48.784626 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21118e0f-62db-4eb4-9edd-25a33cb8a522 service neutron] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:48.798969 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1240/2477] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:51:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:51:48.837471 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cd2de9-006d-476a-bce4-373bbd3ea171 None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:48.838976 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-87cd2de9-006d-476a-bce4-373bbd3ea171 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:51:48.852408 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cd2de9-006d-476a-bce4-373bbd3ea171 None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:48.857415 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf4ffa6-f276-43c7-be93-a51aace89159 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:48.858050 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf4ffa6-f276-43c7-be93-a51aace89159 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:48.858324 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf4ffa6-f276-43c7-be93-a51aace89159 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:48.858594 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf4ffa6-f276-43c7-be93-a51aace89159 admin admin] PATH_INFO: `/v3/users/ac81d76770744c05aca25046fc00ebe2` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:48.859881 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf4ffa6-f276-43c7-be93-a51aace89159 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ac81d76770744c05aca25046fc00ebe2)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:48.862116 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf4ffa6-f276-43c7-be93-a51aace89159 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:49.153382 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-3cf4ffa6-f276-43c7-be93-a51aace89159 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ac81d76770744c05aca25046fc00ebe2'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:49.160957 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-3cf4ffa6-f276-43c7-be93-a51aace89159 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ac81d76770744c05aca25046fc00ebe2'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:49.161636 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-3cf4ffa6-f276-43c7-be93-a51aace89159 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ac81d76770744c05aca25046fc00ebe2'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:49.162368 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-3cf4ffa6-f276-43c7-be93-a51aace89159 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ac81d76770744c05aca25046fc00ebe2'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:49.277237 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-3cf4ffa6-f276-43c7-be93-a51aace89159 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ac81d76770744c05aca25046fc00ebe2'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:49.295184 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1238/2478] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Aug 24 06:51:48 2022] DELETE /identity/v3/users/ac81d76770744c05aca25046fc00ebe2 => generated 0 bytes in 458 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:51:49.299798 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d31127d-568e-4afd-9d02-2dffc28e64b6 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:49.300307 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d31127d-568e-4afd-9d02-2dffc28e64b6 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:49.300413 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d31127d-568e-4afd-9d02-2dffc28e64b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:49.314192 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf4ffa6-f276-43c7-be93-a51aace89159 admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:49.316675 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-3cf4ffa6-f276-43c7-be93-a51aace89159 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:51:49.321144 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-9d31127d-568e-4afd-9d02-2dffc28e64b6 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:51:49.322366 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-9d31127d-568e-4afd-9d02-2dffc28e64b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:51:49.390237 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1241/2479] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:51:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:51:49.397767 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf4ffa6-f276-43c7-be93-a51aace89159 admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:49.400007 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d31127d-568e-4afd-9d02-2dffc28e64b6 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:49.401525 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-9d31127d-568e-4afd-9d02-2dffc28e64b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:51:49.402616 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791500b3-b6cb-4fa1-bff8-036a82335f99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:49.403253 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791500b3-b6cb-4fa1-bff8-036a82335f99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:49.403531 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791500b3-b6cb-4fa1-bff8-036a82335f99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:49.403810 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791500b3-b6cb-4fa1-bff8-036a82335f99 admin admin] PATH_INFO: `/v3/projects/4f70d27dc57741c491584a94951c1d02` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:49.404972 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791500b3-b6cb-4fa1-bff8-036a82335f99 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4f70d27dc57741c491584a94951c1d02)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:49.406911 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791500b3-b6cb-4fa1-bff8-036a82335f99 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:49.419622 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:51:49.425042 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:51:49.463384 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d31127d-568e-4afd-9d02-2dffc28e64b6 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:49.467113 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474fdd20-419d-4e69-b8e4-018b235d538e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bb6c79db75e4d169331c02133d1ec32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:49.467710 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474fdd20-419d-4e69-b8e4-018b235d538e service neutron] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:49.467982 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474fdd20-419d-4e69-b8e4-018b235d538e service neutron] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:49.468325 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474fdd20-419d-4e69-b8e4-018b235d538e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:49.477397 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474fdd20-419d-4e69-b8e4-018b235d538e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:49.478818 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474fdd20-419d-4e69-b8e4-018b235d538e service neutron] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:49.489402 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1242/2480] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:51:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 06:51:49.551898 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-791500b3-b6cb-4fa1-bff8-036a82335f99 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4f70d27dc57741c491584a94951c1d02', 'request_id': 'req-791500b3-b6cb-4fa1-bff8-036a82335f99'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:49.557422 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-791500b3-b6cb-4fa1-bff8-036a82335f99 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4f70d27dc57741c491584a94951c1d02', 'request_id': 'req-791500b3-b6cb-4fa1-bff8-036a82335f99'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:49.565175 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-791500b3-b6cb-4fa1-bff8-036a82335f99 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4f70d27dc57741c491584a94951c1d02', 'request_id': 'req-791500b3-b6cb-4fa1-bff8-036a82335f99'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:49.717530 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-791500b3-b6cb-4fa1-bff8-036a82335f99 admin admin] The token cache is being invalidate because project 4f70d27dc57741c491584a94951c1d02 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81851) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:51:49.719679 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-791500b3-b6cb-4fa1-bff8-036a82335f99 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:49.719679 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-791500b3-b6cb-4fa1-bff8-036a82335f99 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:49.720240 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1239/2481] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Aug 24 06:51:49 2022] DELETE /identity/v3/projects/4f70d27dc57741c491584a94951c1d02 => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:51:49.724374 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474fdd20-419d-4e69-b8e4-018b235d538e service neutron] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:49.737714 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474fdd20-419d-4e69-b8e4-018b235d538e service neutron] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:49.826595 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4c4efe-7155-4b32-900e-e12b5159874f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:49.830597 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4c4efe-7155-4b32-900e-e12b5159874f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:49.830597 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4c4efe-7155-4b32-900e-e12b5159874f admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:49.830597 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4c4efe-7155-4b32-900e-e12b5159874f admin admin] PATH_INFO: `/v3/users/8211405d9e7449fb88a2953c0b4d6bd6` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:49.830597 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4c4efe-7155-4b32-900e-e12b5159874f admin admin] RBAC: Authorizing `identity:delete_user(user_id=8211405d9e7449fb88a2953c0b4d6bd6)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:49.831320 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4c4efe-7155-4b32-900e-e12b5159874f admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:49.983939 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791500b3-b6cb-4fa1-bff8-036a82335f99 admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:49.983939 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-791500b3-b6cb-4fa1-bff8-036a82335f99 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:51:50.000912 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791500b3-b6cb-4fa1-bff8-036a82335f99 admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:50.007398 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8ee41c-b0f2-4c26-9377-d7416fcb3029 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:50.008168 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8ee41c-b0f2-4c26-9377-d7416fcb3029 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:50.008308 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8ee41c-b0f2-4c26-9377-d7416fcb3029 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:50.008628 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8ee41c-b0f2-4c26-9377-d7416fcb3029 admin admin] PATH_INFO: `/v3/projects/58bc554786794fffb848e3db89b979e7` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:50.009886 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8ee41c-b0f2-4c26-9377-d7416fcb3029 admin admin] RBAC: Authorizing `identity:delete_project(project_id=58bc554786794fffb848e3db89b979e7)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:50.011816 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8ee41c-b0f2-4c26-9377-d7416fcb3029 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:50.084558 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-1c4c4efe-7155-4b32-900e-e12b5159874f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8211405d9e7449fb88a2953c0b4d6bd6'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:50.092031 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-1c4c4efe-7155-4b32-900e-e12b5159874f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8211405d9e7449fb88a2953c0b4d6bd6'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:50.092951 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-1c4c4efe-7155-4b32-900e-e12b5159874f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8211405d9e7449fb88a2953c0b4d6bd6'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:50.151663 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-dd8ee41c-b0f2-4c26-9377-d7416fcb3029 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '58bc554786794fffb848e3db89b979e7', 'request_id': 'req-dd8ee41c-b0f2-4c26-9377-d7416fcb3029'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:50.158105 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-1c4c4efe-7155-4b32-900e-e12b5159874f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8211405d9e7449fb88a2953c0b4d6bd6'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:50.158288 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-dd8ee41c-b0f2-4c26-9377-d7416fcb3029 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '58bc554786794fffb848e3db89b979e7', 'request_id': 'req-dd8ee41c-b0f2-4c26-9377-d7416fcb3029'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:50.164990 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-dd8ee41c-b0f2-4c26-9377-d7416fcb3029 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '58bc554786794fffb848e3db89b979e7', 'request_id': 'req-dd8ee41c-b0f2-4c26-9377-d7416fcb3029'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:50.171923 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-1c4c4efe-7155-4b32-900e-e12b5159874f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8211405d9e7449fb88a2953c0b4d6bd6'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:50.178017 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1243/2482] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Aug 24 06:51:49 2022] DELETE /identity/v3/users/8211405d9e7449fb88a2953c0b4d6bd6 => generated 0 bytes in 454 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 06:51:50.240058 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-dd8ee41c-b0f2-4c26-9377-d7416fcb3029 admin admin] The token cache is being invalidate because project 58bc554786794fffb848e3db89b979e7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81851) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:51:50.240393 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-dd8ee41c-b0f2-4c26-9377-d7416fcb3029 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:50.241142 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-dd8ee41c-b0f2-4c26-9377-d7416fcb3029 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:50.242696 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1240/2483] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Aug 24 06:51:49 2022] DELETE /identity/v3/projects/58bc554786794fffb848e3db89b979e7 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 06:51:50.247801 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4c4efe-7155-4b32-900e-e12b5159874f admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:50.453933 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-1c4c4efe-7155-4b32-900e-e12b5159874f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:51:50.453933 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:51:50.453933 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4c4efe-7155-4b32-900e-e12b5159874f admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:50.453933 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed970b4-5c8e-4d03-8fa6-c709310e5470 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:50.457803 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed970b4-5c8e-4d03-8fa6-c709310e5470 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:50.457803 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed970b4-5c8e-4d03-8fa6-c709310e5470 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:50.457803 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed970b4-5c8e-4d03-8fa6-c709310e5470 admin admin] PATH_INFO: `/v3/users/b779cf208fb047659628ec97051ebeb9` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:50.457803 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed970b4-5c8e-4d03-8fa6-c709310e5470 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b779cf208fb047659628ec97051ebeb9)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:50.457803 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed970b4-5c8e-4d03-8fa6-c709310e5470 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:50.711063 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8ee41c-b0f2-4c26-9377-d7416fcb3029 admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:50.712781 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-dd8ee41c-b0f2-4c26-9377-d7416fcb3029 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:51:50.728980 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-1ed970b4-5c8e-4d03-8fa6-c709310e5470 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b779cf208fb047659628ec97051ebeb9'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:50.739232 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-1ed970b4-5c8e-4d03-8fa6-c709310e5470 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b779cf208fb047659628ec97051ebeb9'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:50.740110 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-1ed970b4-5c8e-4d03-8fa6-c709310e5470 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b779cf208fb047659628ec97051ebeb9'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:50.790699 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8ee41c-b0f2-4c26-9377-d7416fcb3029 admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:50.795496 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894acb2d-d440-4f18-8295-da01ab92b791 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:50.796147 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894acb2d-d440-4f18-8295-da01ab92b791 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:50.796523 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894acb2d-d440-4f18-8295-da01ab92b791 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:50.796724 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894acb2d-d440-4f18-8295-da01ab92b791 admin admin] PATH_INFO: `/v3/projects/6b0f57e7b36d418f9cd961ccae41ad01` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:50.797854 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894acb2d-d440-4f18-8295-da01ab92b791 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6b0f57e7b36d418f9cd961ccae41ad01)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:50.799947 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894acb2d-d440-4f18-8295-da01ab92b791 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:50.834645 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:51:50.862590 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-1ed970b4-5c8e-4d03-8fa6-c709310e5470 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b779cf208fb047659628ec97051ebeb9'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:50.877581 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-1ed970b4-5c8e-4d03-8fa6-c709310e5470 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b779cf208fb047659628ec97051ebeb9'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:50.884000 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1244/2484] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Aug 24 06:51:50 2022] DELETE /identity/v3/users/b779cf208fb047659628ec97051ebeb9 => generated 0 bytes in 638 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:51:50.942163 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-894acb2d-d440-4f18-8295-da01ab92b791 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6b0f57e7b36d418f9cd961ccae41ad01', 'request_id': 'req-894acb2d-d440-4f18-8295-da01ab92b791'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:50.947687 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-894acb2d-d440-4f18-8295-da01ab92b791 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6b0f57e7b36d418f9cd961ccae41ad01', 'request_id': 'req-894acb2d-d440-4f18-8295-da01ab92b791'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:50.954234 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-894acb2d-d440-4f18-8295-da01ab92b791 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6b0f57e7b36d418f9cd961ccae41ad01', 'request_id': 'req-894acb2d-d440-4f18-8295-da01ab92b791'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:51.104436 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-894acb2d-d440-4f18-8295-da01ab92b791 admin admin] The token cache is being invalidate because project 6b0f57e7b36d418f9cd961ccae41ad01 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81851) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:51:51.104877 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-894acb2d-d440-4f18-8295-da01ab92b791 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:51.106195 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-894acb2d-d440-4f18-8295-da01ab92b791 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:51.108024 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1241/2485] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Aug 24 06:51:50 2022] DELETE /identity/v3/projects/6b0f57e7b36d418f9cd961ccae41ad01 => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 06:51:51.113738 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464fad37-70e1-4964-9ccd-2f64c179b289 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:51.114090 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464fad37-70e1-4964-9ccd-2f64c179b289 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:51.114370 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464fad37-70e1-4964-9ccd-2f64c179b289 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:51.139932 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-464fad37-70e1-4964-9ccd-2f64c179b289 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:51:51.140104 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-464fad37-70e1-4964-9ccd-2f64c179b289 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:51:51.216748 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1245/2486] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:51:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:51:51.224742 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eddef93-3615-4d38-a04f-27393380ee08 None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:51.227130 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eddef93-3615-4d38-a04f-27393380ee08 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:51.227130 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eddef93-3615-4d38-a04f-27393380ee08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:51.247491 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-1eddef93-3615-4d38-a04f-27393380ee08 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:51:51.248664 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-1eddef93-3615-4d38-a04f-27393380ee08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:51:51.257197 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1242/2487] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:51:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:51:51.262937 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464fad37-70e1-4964-9ccd-2f64c179b289 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:51.273387 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464fad37-70e1-4964-9ccd-2f64c179b289 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:51.278875 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e5184a-0d10-405f-9a35-9607ed174148 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:51.279481 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e5184a-0d10-405f-9a35-9607ed174148 admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:51.279756 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e5184a-0d10-405f-9a35-9607ed174148 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:51.280023 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e5184a-0d10-405f-9a35-9607ed174148 admin admin] PATH_INFO: `/v3/domains` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:51.280694 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e5184a-0d10-405f-9a35-9607ed174148 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:51:51.280993 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e5184a-0d10-405f-9a35-9607ed174148 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:51.282605 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e5184a-0d10-405f-9a35-9607ed174148 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:51.295280 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1246/2488] 10.222.0.62 () {58 vars in 1153 bytes} [Wed Aug 24 06:51:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:51:51.299710 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eddef93-3615-4d38-a04f-27393380ee08 None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:51.308983 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eddef93-3615-4d38-a04f-27393380ee08 None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:51.314042 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a9fb64-13a7-4161-8837-38399bb81bbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:51.315417 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a9fb64-13a7-4161-8837-38399bb81bbc admin admin] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:51.315712 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a9fb64-13a7-4161-8837-38399bb81bbc admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:51.315995 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a9fb64-13a7-4161-8837-38399bb81bbc admin admin] PATH_INFO: `/v3/projects` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:51.316670 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a9fb64-13a7-4161-8837-38399bb81bbc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:51.318724 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a9fb64-13a7-4161-8837-38399bb81bbc admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:51.385245 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e5184a-0d10-405f-9a35-9607ed174148 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:51.387205 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-16e5184a-0d10-405f-9a35-9607ed174148 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:51:51.401881 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e5184a-0d10-405f-9a35-9607ed174148 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:51.407402 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da853c51-9e45-4b5b-ad49-1958ba852e9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:51.408048 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da853c51-9e45-4b5b-ad49-1958ba852e9a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:51.408329 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da853c51-9e45-4b5b-ad49-1958ba852e9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:51.408636 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da853c51-9e45-4b5b-ad49-1958ba852e9a admin admin] PATH_INFO: `/v3/projects/82913b80c8224c7c9438272bc1ee11f2` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:51.409831 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da853c51-9e45-4b5b-ad49-1958ba852e9a admin admin] RBAC: Authorizing `identity:delete_project(project_id=82913b80c8224c7c9438272bc1ee11f2)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:51.419588 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da853c51-9e45-4b5b-ad49-1958ba852e9a admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:51.446933 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1243/2489] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Aug 24 06:51:51 2022] POST /identity/v3/projects => generated 377 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:51:51.450980 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a9fb64-13a7-4161-8837-38399bb81bbc admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:51.516589 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a9fb64-13a7-4161-8837-38399bb81bbc admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:51.521467 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723af396-7d5b-4450-96a0-70fa7ee8e616 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:51.522075 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723af396-7d5b-4450-96a0-70fa7ee8e616 admin admin] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:51.522355 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723af396-7d5b-4450-96a0-70fa7ee8e616 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:51.522626 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723af396-7d5b-4450-96a0-70fa7ee8e616 admin admin] PATH_INFO: `/v3/users` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:51.523245 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723af396-7d5b-4450-96a0-70fa7ee8e616 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:51.525278 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723af396-7d5b-4450-96a0-70fa7ee8e616 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:51.595768 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:51:51.602054 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-da853c51-9e45-4b5b-ad49-1958ba852e9a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '82913b80c8224c7c9438272bc1ee11f2', 'request_id': 'req-da853c51-9e45-4b5b-ad49-1958ba852e9a'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:51.608110 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-da853c51-9e45-4b5b-ad49-1958ba852e9a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '82913b80c8224c7c9438272bc1ee11f2', 'request_id': 'req-da853c51-9e45-4b5b-ad49-1958ba852e9a'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:51.614808 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-da853c51-9e45-4b5b-ad49-1958ba852e9a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '82913b80c8224c7c9438272bc1ee11f2', 'request_id': 'req-da853c51-9e45-4b5b-ad49-1958ba852e9a'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:51.631789 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1244/2490] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Aug 24 06:51:51 2022] POST /identity/v3/users => generated 289 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 06:51:51.631789 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723af396-7d5b-4450-96a0-70fa7ee8e616 admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:51.641488 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723af396-7d5b-4450-96a0-70fa7ee8e616 admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:51.702259 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac2b78a-1002-4fa0-a872-2eee54a9f0ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:51.702879 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac2b78a-1002-4fa0-a872-2eee54a9f0ea admin admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:51.703241 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac2b78a-1002-4fa0-a872-2eee54a9f0ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:51.703430 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac2b78a-1002-4fa0-a872-2eee54a9f0ea admin admin] PATH_INFO: `/v3/roles` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:51.704125 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac2b78a-1002-4fa0-a872-2eee54a9f0ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:51.705761 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac2b78a-1002-4fa0-a872-2eee54a9f0ea admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:51.717237 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1245/2491] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Aug 24 06:51:51 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 06:51:51.731480 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac2b78a-1002-4fa0-a872-2eee54a9f0ea admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:51.742141 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac2b78a-1002-4fa0-a872-2eee54a9f0ea admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:51.747548 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd3a95b-a006-4f55-b4ed-d0e0683b129b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:51.748183 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd3a95b-a006-4f55-b4ed-d0e0683b129b admin admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:51.748460 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd3a95b-a006-4f55-b4ed-d0e0683b129b admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:51.748758 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd3a95b-a006-4f55-b4ed-d0e0683b129b admin admin] PATH_INFO: `/v3/roles` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:51.749394 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd3a95b-a006-4f55-b4ed-d0e0683b129b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:51.750992 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd3a95b-a006-4f55-b4ed-d0e0683b129b admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:51.753437 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=89,keystone:DELETE=6,keystone:INSERT=5 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:51:51.762772 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1246/2492] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Aug 24 06:51:51 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 06:51:51.766867 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd3a95b-a006-4f55-b4ed-d0e0683b129b admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:51.768975 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-da853c51-9e45-4b5b-ad49-1958ba852e9a admin admin] The token cache is being invalidate because project 82913b80c8224c7c9438272bc1ee11f2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81852) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:51:51.769268 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-da853c51-9e45-4b5b-ad49-1958ba852e9a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:51.769926 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-da853c51-9e45-4b5b-ad49-1958ba852e9a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:51.771764 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1247/2493] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Aug 24 06:51:51 2022] DELETE /identity/v3/projects/82913b80c8224c7c9438272bc1ee11f2 => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 06:51:51.775679 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da853c51-9e45-4b5b-ad49-1958ba852e9a admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:51.777092 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-da853c51-9e45-4b5b-ad49-1958ba852e9a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:51:51.777797 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd3a95b-a006-4f55-b4ed-d0e0683b129b admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:51.782639 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8866ec1b-03f3-46a4-93bc-9d55b9e1c179 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:51.783303 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8866ec1b-03f3-46a4-93bc-9d55b9e1c179 admin admin] REQUEST_METHOD: `PUT` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:51.783576 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8866ec1b-03f3-46a4-93bc-9d55b9e1c179 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:51.783847 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8866ec1b-03f3-46a4-93bc-9d55b9e1c179 admin admin] PATH_INFO: `/v3/projects/4c625f068e0e4f2e8df02672171adb72/users/bcd35ac12b4145419178e7d538ba8b0e/roles/580aa7dbb38d4cff8f9d1bb8983f99ef` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:51.790165 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da853c51-9e45-4b5b-ad49-1958ba852e9a admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:51.795506 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b59b60-3a1a-448a-a03c-940114818feb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:51.796171 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b59b60-3a1a-448a-a03c-940114818feb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:51.796484 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b59b60-3a1a-448a-a03c-940114818feb admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:51.796787 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b59b60-3a1a-448a-a03c-940114818feb admin admin] PATH_INFO: `/v3/users/f4e465232d5146019cea15f56494af22` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:51.797960 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b59b60-3a1a-448a-a03c-940114818feb admin admin] RBAC: Authorizing `identity:delete_user(user_id=f4e465232d5146019cea15f56494af22)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:51.799964 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b59b60-3a1a-448a-a03c-940114818feb admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:51.802774 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8866ec1b-03f3-46a4-93bc-9d55b9e1c179 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c625f068e0e4f2e8df02672171adb72, user_id=bcd35ac12b4145419178e7d538ba8b0e, role_id=580aa7dbb38d4cff8f9d1bb8983f99ef)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:51.802774 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8866ec1b-03f3-46a4-93bc-9d55b9e1c179 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:51.904088 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1247/2494] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Aug 24 06:51:51 2022] PUT /identity/v3/projects/4c625f068e0e4f2e8df02672171adb72/users/bcd35ac12b4145419178e7d538ba8b0e/roles/580aa7dbb38d4cff8f9d1bb8983f99ef => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 06:51:51.974261 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1695d862-a7d5-4179-b60d-19a80cde5ed1 None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:51.974555 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1695d862-a7d5-4179-b60d-19a80cde5ed1 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:51.974836 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1695d862-a7d5-4179-b60d-19a80cde5ed1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:52.003893 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-04b59b60-3a1a-448a-a03c-940114818feb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f4e465232d5146019cea15f56494af22'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:52.005891 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-1695d862-a7d5-4179-b60d-19a80cde5ed1 None None] MFA Rules not processed for user `bcd35ac12b4145419178e7d538ba8b0e`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:51:52.007120 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-1695d862-a7d5-4179-b60d-19a80cde5ed1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:51:52.011204 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-04b59b60-3a1a-448a-a03c-940114818feb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f4e465232d5146019cea15f56494af22'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:52.011920 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-04b59b60-3a1a-448a-a03c-940114818feb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f4e465232d5146019cea15f56494af22'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:52.070955 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1248/2495] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:51:51 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 06:51:52.082947 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1695d862-a7d5-4179-b60d-19a80cde5ed1 None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:52.084284 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-1695d862-a7d5-4179-b60d-19a80cde5ed1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:51:52.091703 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-04b59b60-3a1a-448a-a03c-940114818feb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f4e465232d5146019cea15f56494af22'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:52.104776 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-04b59b60-3a1a-448a-a03c-940114818feb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f4e465232d5146019cea15f56494af22'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:52.111606 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1248/2496] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Aug 24 06:51:51 2022] DELETE /identity/v3/users/f4e465232d5146019cea15f56494af22 => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:51:52.146075 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1695d862-a7d5-4179-b60d-19a80cde5ed1 None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:52.197296 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da517bcb-e449-4fc9-aa8f-95224f3b8e80 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7d52162cbdc4218b577ecc1e69a88c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:52.197919 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da517bcb-e449-4fc9-aa8f-95224f3b8e80 service glance] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:52.198201 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da517bcb-e449-4fc9-aa8f-95224f3b8e80 service glance] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:52.198476 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da517bcb-e449-4fc9-aa8f-95224f3b8e80 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:52.200309 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-da517bcb-e449-4fc9-aa8f-95224f3b8e80 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:51:52.236158 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:51:52.263103 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da517bcb-e449-4fc9-aa8f-95224f3b8e80 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:52.264648 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da517bcb-e449-4fc9-aa8f-95224f3b8e80 service glance] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:52.273624 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1249/2497] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:51:52 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 06:51:52.622100 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b59b60-3a1a-448a-a03c-940114818feb admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:52.624125 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-04b59b60-3a1a-448a-a03c-940114818feb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:51:52.698434 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b59b60-3a1a-448a-a03c-940114818feb admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:52.710267 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d72cdc-a581-46e3-83d6-140074485079 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:52.710931 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d72cdc-a581-46e3-83d6-140074485079 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:52.711213 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d72cdc-a581-46e3-83d6-140074485079 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:52.711485 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d72cdc-a581-46e3-83d6-140074485079 admin admin] PATH_INFO: `/v3/projects/b0157f47a7464a7d87f4005c6670fd81` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:52.712702 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d72cdc-a581-46e3-83d6-140074485079 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b0157f47a7464a7d87f4005c6670fd81)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:52.714621 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d72cdc-a581-46e3-83d6-140074485079 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:52.900204 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:51:52.903812 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-38d72cdc-a581-46e3-83d6-140074485079 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b0157f47a7464a7d87f4005c6670fd81', 'request_id': 'req-38d72cdc-a581-46e3-83d6-140074485079'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:52.909652 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-38d72cdc-a581-46e3-83d6-140074485079 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b0157f47a7464a7d87f4005c6670fd81', 'request_id': 'req-38d72cdc-a581-46e3-83d6-140074485079'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:52.915824 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-38d72cdc-a581-46e3-83d6-140074485079 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b0157f47a7464a7d87f4005c6670fd81', 'request_id': 'req-38d72cdc-a581-46e3-83d6-140074485079'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:53.031232 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-38d72cdc-a581-46e3-83d6-140074485079 admin admin] The token cache is being invalidate because project b0157f47a7464a7d87f4005c6670fd81 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81852) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:51:53.031609 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-38d72cdc-a581-46e3-83d6-140074485079 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:53.032329 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-38d72cdc-a581-46e3-83d6-140074485079 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:53.034098 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1249/2498] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Aug 24 06:51:52 2022] DELETE /identity/v3/projects/b0157f47a7464a7d87f4005c6670fd81 => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:51:53.040350 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebab422-fa21-43f1-aeb7-9354dc2b885e None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:53.040846 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebab422-fa21-43f1-aeb7-9354dc2b885e None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:53.040975 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebab422-fa21-43f1-aeb7-9354dc2b885e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:53.063050 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-6ebab422-fa21-43f1-aeb7-9354dc2b885e None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:51:53.064284 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-6ebab422-fa21-43f1-aeb7-9354dc2b885e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:51:53.126680 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1250/2499] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:51:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:51:53.134968 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7cf7dc-7f7f-498b-a2a8-f6c911d5c90d None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:53.135253 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7cf7dc-7f7f-498b-a2a8-f6c911d5c90d None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:53.135527 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7cf7dc-7f7f-498b-a2a8-f6c911d5c90d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:53.158067 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-7e7cf7dc-7f7f-498b-a2a8-f6c911d5c90d None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:51:53.159204 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-7e7cf7dc-7f7f-498b-a2a8-f6c911d5c90d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:51:53.168441 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1250/2500] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:51:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:51:53.172652 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebab422-fa21-43f1-aeb7-9354dc2b885e None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:53.182784 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebab422-fa21-43f1-aeb7-9354dc2b885e None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:53.188306 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b387d6f-8c43-4e9d-807e-fca8473fbb83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:53.188935 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b387d6f-8c43-4e9d-807e-fca8473fbb83 admin admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:53.189237 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b387d6f-8c43-4e9d-807e-fca8473fbb83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:53.189637 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b387d6f-8c43-4e9d-807e-fca8473fbb83 admin admin] PATH_INFO: `/v3/domains` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:53.190277 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b387d6f-8c43-4e9d-807e-fca8473fbb83 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:51:53.190576 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b387d6f-8c43-4e9d-807e-fca8473fbb83 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:53.192161 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b387d6f-8c43-4e9d-807e-fca8473fbb83 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:53.204108 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1251/2501] 10.222.0.62 () {58 vars in 1153 bytes} [Wed Aug 24 06:51:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:51:53.210839 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7cf7dc-7f7f-498b-a2a8-f6c911d5c90d None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:53.220418 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7cf7dc-7f7f-498b-a2a8-f6c911d5c90d None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:53.225613 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbaf80d5-bfb7-47bd-8214-32513f89e307 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:53.226233 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbaf80d5-bfb7-47bd-8214-32513f89e307 admin admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:53.226512 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbaf80d5-bfb7-47bd-8214-32513f89e307 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:53.226782 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbaf80d5-bfb7-47bd-8214-32513f89e307 admin admin] PATH_INFO: `/v3/projects` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:53.227421 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbaf80d5-bfb7-47bd-8214-32513f89e307 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:53.229399 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbaf80d5-bfb7-47bd-8214-32513f89e307 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:53.334414 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1251/2502] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Aug 24 06:51:53 2022] POST /identity/v3/projects => generated 379 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:51:53.338672 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b387d6f-8c43-4e9d-807e-fca8473fbb83 admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:53.349300 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b387d6f-8c43-4e9d-807e-fca8473fbb83 admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:53.415303 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c55b08-49c1-4ac1-b61e-bc4e972ece29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:53.415923 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c55b08-49c1-4ac1-b61e-bc4e972ece29 admin admin] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:53.416244 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c55b08-49c1-4ac1-b61e-bc4e972ece29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:53.416470 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c55b08-49c1-4ac1-b61e-bc4e972ece29 admin admin] PATH_INFO: `/v3/users` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:53.417120 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c55b08-49c1-4ac1-b61e-bc4e972ece29 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:53.419605 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c55b08-49c1-4ac1-b61e-bc4e972ece29 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:53.574532 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1252/2503] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Aug 24 06:51:53 2022] POST /identity/v3/users => generated 290 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:51:53.578872 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbaf80d5-bfb7-47bd-8214-32513f89e307 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:53.590381 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbaf80d5-bfb7-47bd-8214-32513f89e307 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:53.595112 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40533afa-2074-4696-925f-107bb15e6f3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:53.595730 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40533afa-2074-4696-925f-107bb15e6f3d admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:53.596006 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40533afa-2074-4696-925f-107bb15e6f3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:53.597044 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40533afa-2074-4696-925f-107bb15e6f3d admin admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:53.597044 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40533afa-2074-4696-925f-107bb15e6f3d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:53.598631 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40533afa-2074-4696-925f-107bb15e6f3d admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:53.610638 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1252/2504] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Aug 24 06:51:53 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:51:53.615011 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c55b08-49c1-4ac1-b61e-bc4e972ece29 admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:53.625373 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c55b08-49c1-4ac1-b61e-bc4e972ece29 admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:53.630352 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1ce61b-e731-461c-a3aa-03a098eb240b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:53.630956 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1ce61b-e731-461c-a3aa-03a098eb240b admin admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:53.631312 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1ce61b-e731-461c-a3aa-03a098eb240b admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:53.631506 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1ce61b-e731-461c-a3aa-03a098eb240b admin admin] PATH_INFO: `/v3/roles` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:53.632146 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1ce61b-e731-461c-a3aa-03a098eb240b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:53.633794 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1ce61b-e731-461c-a3aa-03a098eb240b admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:53.644856 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1253/2505] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Aug 24 06:51:53 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:51:53.649158 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40533afa-2074-4696-925f-107bb15e6f3d admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:53.660024 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40533afa-2074-4696-925f-107bb15e6f3d admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:53.664735 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10762bb1-170b-4cfd-ba5a-a15b8a639a04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:53.665384 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10762bb1-170b-4cfd-ba5a-a15b8a639a04 admin admin] REQUEST_METHOD: `PUT` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:53.665660 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10762bb1-170b-4cfd-ba5a-a15b8a639a04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:53.665990 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10762bb1-170b-4cfd-ba5a-a15b8a639a04 admin admin] PATH_INFO: `/v3/projects/834d5c0f457b4a24b72bb098ec6193ae/users/e97dd542122e41eaa7618247573433aa/roles/580aa7dbb38d4cff8f9d1bb8983f99ef` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:53.682022 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10762bb1-170b-4cfd-ba5a-a15b8a639a04 admin admin] RBAC: Authorizing `identity:create_grant(project_id=834d5c0f457b4a24b72bb098ec6193ae, user_id=e97dd542122e41eaa7618247573433aa, role_id=580aa7dbb38d4cff8f9d1bb8983f99ef)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:53.684839 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10762bb1-170b-4cfd-ba5a-a15b8a639a04 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:53.762944 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1253/2506] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Aug 24 06:51:53 2022] PUT /identity/v3/projects/834d5c0f457b4a24b72bb098ec6193ae/users/e97dd542122e41eaa7618247573433aa/roles/580aa7dbb38d4cff8f9d1bb8983f99ef => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:51:53.775455 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34b7ae7-ef6b-4456-9544-d68ab12e390f None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:53.775740 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34b7ae7-ef6b-4456-9544-d68ab12e390f None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:53.776013 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34b7ae7-ef6b-4456-9544-d68ab12e390f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:53.804579 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-a34b7ae7-ef6b-4456-9544-d68ab12e390f None None] MFA Rules not processed for user `e97dd542122e41eaa7618247573433aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:51:53.805757 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-a34b7ae7-ef6b-4456-9544-d68ab12e390f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:51:53.834397 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:51:53.870481 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1254/2507] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:51:53 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:51:53.874748 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10762bb1-170b-4cfd-ba5a-a15b8a639a04 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:53.937061 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10762bb1-170b-4cfd-ba5a-a15b8a639a04 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:53.941655 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9f9162-c986-41e3-9b49-c386d787a343 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:53.942252 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9f9162-c986-41e3-9b49-c386d787a343 admin admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:53.942526 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9f9162-c986-41e3-9b49-c386d787a343 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:53.942792 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9f9162-c986-41e3-9b49-c386d787a343 admin admin] PATH_INFO: `/v3/projects` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:53.943434 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9f9162-c986-41e3-9b49-c386d787a343 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:53.945377 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9f9162-c986-41e3-9b49-c386d787a343 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:54.039006 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1254/2508] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Aug 24 06:51:53 2022] POST /identity/v3/projects => generated 379 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:51:54.043018 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34b7ae7-ef6b-4456-9544-d68ab12e390f None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:54.053972 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34b7ae7-ef6b-4456-9544-d68ab12e390f None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:54.112522 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efe0703-0f8a-43ad-838d-0c1dd5f6cd47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:54.113122 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efe0703-0f8a-43ad-838d-0c1dd5f6cd47 admin admin] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:54.113399 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efe0703-0f8a-43ad-838d-0c1dd5f6cd47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:54.113668 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efe0703-0f8a-43ad-838d-0c1dd5f6cd47 admin admin] PATH_INFO: `/v3/users` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:54.114284 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efe0703-0f8a-43ad-838d-0c1dd5f6cd47 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:54.116193 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efe0703-0f8a-43ad-838d-0c1dd5f6cd47 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:54.188850 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1255/2509] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Aug 24 06:51:54 2022] POST /identity/v3/users => generated 290 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:51:54.192985 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9f9162-c986-41e3-9b49-c386d787a343 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:54.203937 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9f9162-c986-41e3-9b49-c386d787a343 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:54.208484 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23899254-ba0e-4c9c-b7b6-0ab4f3ccb981 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:54.209101 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23899254-ba0e-4c9c-b7b6-0ab4f3ccb981 admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:54.209377 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23899254-ba0e-4c9c-b7b6-0ab4f3ccb981 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:54.209660 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23899254-ba0e-4c9c-b7b6-0ab4f3ccb981 admin admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:54.210315 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23899254-ba0e-4c9c-b7b6-0ab4f3ccb981 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:54.211891 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23899254-ba0e-4c9c-b7b6-0ab4f3ccb981 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:54.223108 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1255/2510] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Aug 24 06:51:54 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:51:54.227337 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efe0703-0f8a-43ad-838d-0c1dd5f6cd47 admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:54.237375 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efe0703-0f8a-43ad-838d-0c1dd5f6cd47 admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:54.241935 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e67784d-798e-463b-8075-7b8ec49921fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:54.242526 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e67784d-798e-463b-8075-7b8ec49921fc admin admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:54.242803 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e67784d-798e-463b-8075-7b8ec49921fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:54.243074 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e67784d-798e-463b-8075-7b8ec49921fc admin admin] PATH_INFO: `/v3/roles` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:54.243705 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e67784d-798e-463b-8075-7b8ec49921fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:54.245398 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e67784d-798e-463b-8075-7b8ec49921fc admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:54.255263 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1256/2511] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Aug 24 06:51:54 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:51:54.259371 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23899254-ba0e-4c9c-b7b6-0ab4f3ccb981 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:54.269925 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23899254-ba0e-4c9c-b7b6-0ab4f3ccb981 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:54.274421 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef31d8b-0a07-449d-a82c-748fa15538f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:54.275060 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef31d8b-0a07-449d-a82c-748fa15538f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:54.275342 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef31d8b-0a07-449d-a82c-748fa15538f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:54.275611 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef31d8b-0a07-449d-a82c-748fa15538f7 admin admin] PATH_INFO: `/v3/projects/ee04558220be4e3a9ecc92ea6d3f33c7/users/6e2b58f1c2f1437089ba9b59d53e7c7f/roles/580aa7dbb38d4cff8f9d1bb8983f99ef` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:54.290429 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef31d8b-0a07-449d-a82c-748fa15538f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ee04558220be4e3a9ecc92ea6d3f33c7, user_id=6e2b58f1c2f1437089ba9b59d53e7c7f, role_id=580aa7dbb38d4cff8f9d1bb8983f99ef)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:54.293108 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef31d8b-0a07-449d-a82c-748fa15538f7 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:54.406159 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1256/2512] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Aug 24 06:51:54 2022] PUT /identity/v3/projects/ee04558220be4e3a9ecc92ea6d3f33c7/users/6e2b58f1c2f1437089ba9b59d53e7c7f/roles/580aa7dbb38d4cff8f9d1bb8983f99ef => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:51:54.416261 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb2a23d-5e21-4abf-9f8c-79426ee24ce5 None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:54.416576 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb2a23d-5e21-4abf-9f8c-79426ee24ce5 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:54.416850 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb2a23d-5e21-4abf-9f8c-79426ee24ce5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:54.446508 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-bfb2a23d-5e21-4abf-9f8c-79426ee24ce5 None None] MFA Rules not processed for user `6e2b58f1c2f1437089ba9b59d53e7c7f`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:51:54.447674 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-bfb2a23d-5e21-4abf-9f8c-79426ee24ce5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:51:54.512287 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1257/2513] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:51:54 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:51:54.516798 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef31d8b-0a07-449d-a82c-748fa15538f7 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:54.535921 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=91,keystone:DELETE=4,keystone:INSERT=5 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:51:54.580959 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef31d8b-0a07-449d-a82c-748fa15538f7 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:54.585810 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c33c5b-6d85-4996-9de1-85dea34f1d95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:54.586406 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c33c5b-6d85-4996-9de1-85dea34f1d95 admin admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:54.586683 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c33c5b-6d85-4996-9de1-85dea34f1d95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:54.586952 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c33c5b-6d85-4996-9de1-85dea34f1d95 admin admin] PATH_INFO: `/v3/projects` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:54.587582 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c33c5b-6d85-4996-9de1-85dea34f1d95 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:54.589705 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c33c5b-6d85-4996-9de1-85dea34f1d95 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:54.696447 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1257/2514] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Aug 24 06:51:54 2022] POST /identity/v3/projects => generated 379 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 06:51:54.700695 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb2a23d-5e21-4abf-9f8c-79426ee24ce5 None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:54.713132 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb2a23d-5e21-4abf-9f8c-79426ee24ce5 None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:54.751114 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:51:54.775415 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8609272-0512-48bf-ae06-82f42c1867c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:54.776029 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8609272-0512-48bf-ae06-82f42c1867c7 admin admin] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:54.776308 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8609272-0512-48bf-ae06-82f42c1867c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:54.776617 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8609272-0512-48bf-ae06-82f42c1867c7 admin admin] PATH_INFO: `/v3/users` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:54.777251 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8609272-0512-48bf-ae06-82f42c1867c7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:54.779228 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8609272-0512-48bf-ae06-82f42c1867c7 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:54.869491 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1258/2515] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Aug 24 06:51:54 2022] POST /identity/v3/users => generated 296 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 06:51:54.874063 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c33c5b-6d85-4996-9de1-85dea34f1d95 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:54.884644 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c33c5b-6d85-4996-9de1-85dea34f1d95 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:54.893736 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b87854-1089-421b-a059-34abfcc80da4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:54.893736 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b87854-1089-421b-a059-34abfcc80da4 admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:54.893736 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b87854-1089-421b-a059-34abfcc80da4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:54.894128 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b87854-1089-421b-a059-34abfcc80da4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:54.894593 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b87854-1089-421b-a059-34abfcc80da4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:54.896989 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b87854-1089-421b-a059-34abfcc80da4 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:54.908607 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1258/2516] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Aug 24 06:51:54 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:51:54.913409 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8609272-0512-48bf-ae06-82f42c1867c7 admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:54.926152 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8609272-0512-48bf-ae06-82f42c1867c7 admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:54.931289 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c9066c-407f-4317-95a5-dbb0a52dbdc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:54.931951 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c9066c-407f-4317-95a5-dbb0a52dbdc1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:54.932232 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c9066c-407f-4317-95a5-dbb0a52dbdc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:54.932540 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c9066c-407f-4317-95a5-dbb0a52dbdc1 admin admin] PATH_INFO: `/v3/projects/b04308fbe979463fa9014d1f50051687/users/83f575b5ac08431eb5d0f0ed9d2250f7/roles/49e47cd157d743f6905f1db07ec878a7` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:54.950211 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c9066c-407f-4317-95a5-dbb0a52dbdc1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b04308fbe979463fa9014d1f50051687, user_id=83f575b5ac08431eb5d0f0ed9d2250f7, role_id=49e47cd157d743f6905f1db07ec878a7)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:54.953125 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c9066c-407f-4317-95a5-dbb0a52dbdc1 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:55.054726 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1259/2517] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Aug 24 06:51:54 2022] PUT /identity/v3/projects/b04308fbe979463fa9014d1f50051687/users/83f575b5ac08431eb5d0f0ed9d2250f7/roles/49e47cd157d743f6905f1db07ec878a7 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 06:51:55.059281 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b87854-1089-421b-a059-34abfcc80da4 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:55.070502 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b87854-1089-421b-a059-34abfcc80da4 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:55.140581 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71830bbd-dd26-4125-9b6d-7da9012cd9e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:55.141305 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71830bbd-dd26-4125-9b6d-7da9012cd9e0 admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:55.141558 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71830bbd-dd26-4125-9b6d-7da9012cd9e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:55.141858 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71830bbd-dd26-4125-9b6d-7da9012cd9e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:55.142479 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71830bbd-dd26-4125-9b6d-7da9012cd9e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:55.144213 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71830bbd-dd26-4125-9b6d-7da9012cd9e0 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:55.154566 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1259/2518] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Aug 24 06:51:55 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:51:55.162961 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c9066c-407f-4317-95a5-dbb0a52dbdc1 admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:55.174439 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c9066c-407f-4317-95a5-dbb0a52dbdc1 admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:55.179148 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72570b6f-f3d1-41de-8e84-6c2e90a21d75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:55.179812 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72570b6f-f3d1-41de-8e84-6c2e90a21d75 admin admin] REQUEST_METHOD: `PUT` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:55.180093 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72570b6f-f3d1-41de-8e84-6c2e90a21d75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:55.180368 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72570b6f-f3d1-41de-8e84-6c2e90a21d75 admin admin] PATH_INFO: `/v3/projects/b04308fbe979463fa9014d1f50051687/users/83f575b5ac08431eb5d0f0ed9d2250f7/roles/49e47cd157d743f6905f1db07ec878a7` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:55.182470 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72570b6f-f3d1-41de-8e84-6c2e90a21d75 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b04308fbe979463fa9014d1f50051687, user_id=83f575b5ac08431eb5d0f0ed9d2250f7, role_id=49e47cd157d743f6905f1db07ec878a7)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:55.186780 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72570b6f-f3d1-41de-8e84-6c2e90a21d75 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:55.197930 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1260/2519] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Aug 24 06:51:55 2022] PUT /identity/v3/projects/b04308fbe979463fa9014d1f50051687/users/83f575b5ac08431eb5d0f0ed9d2250f7/roles/49e47cd157d743f6905f1db07ec878a7 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:51:55.208628 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd191c2b-25ea-4ca8-9218-60d90b54f9d7 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:55.208871 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd191c2b-25ea-4ca8-9218-60d90b54f9d7 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:55.209127 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd191c2b-25ea-4ca8-9218-60d90b54f9d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:55.238425 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-cd191c2b-25ea-4ca8-9218-60d90b54f9d7 None None] MFA Rules not processed for user `83f575b5ac08431eb5d0f0ed9d2250f7`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:51:55.239651 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-cd191c2b-25ea-4ca8-9218-60d90b54f9d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:51:55.309651 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1260/2520] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:51:55 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:51:55.321609 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72570b6f-f3d1-41de-8e84-6c2e90a21d75 admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:55.323104 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-72570b6f-f3d1-41de-8e84-6c2e90a21d75 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:51:55.386424 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72570b6f-f3d1-41de-8e84-6c2e90a21d75 admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:55.391108 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8164ccbd-cdf5-4a53-b529-dc91c26cad1d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c4def55ac0e48b29836bc5c64bc7402', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:55.391756 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8164ccbd-cdf5-4a53-b529-dc91c26cad1d service cinder] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:55.392539 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8164ccbd-cdf5-4a53-b529-dc91c26cad1d service cinder] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:55.392876 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8164ccbd-cdf5-4a53-b529-dc91c26cad1d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:55.404761 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8164ccbd-cdf5-4a53-b529-dc91c26cad1d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:55.406255 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8164ccbd-cdf5-4a53-b529-dc91c26cad1d service cinder] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:55.416460 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1261/2521] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:51:55 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:51:55.531916 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd191c2b-25ea-4ca8-9218-60d90b54f9d7 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:55.533403 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-cd191c2b-25ea-4ca8-9218-60d90b54f9d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:51:55.546905 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd191c2b-25ea-4ca8-9218-60d90b54f9d7 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:55.550743 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec28b8c-94f3-413f-ad52-f469fb4532d4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c4def55ac0e48b29836bc5c64bc7402', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:55.551354 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec28b8c-94f3-413f-ad52-f469fb4532d4 service cinder] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:55.551629 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec28b8c-94f3-413f-ad52-f469fb4532d4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:55.551897 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec28b8c-94f3-413f-ad52-f469fb4532d4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:55.563333 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:51:55.615228 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec28b8c-94f3-413f-ad52-f469fb4532d4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:55.617003 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec28b8c-94f3-413f-ad52-f469fb4532d4 service cinder] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:55.627468 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1261/2522] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:51:55 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:51:59.899523 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8164ccbd-cdf5-4a53-b529-dc91c26cad1d service cinder] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:59.963237 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8164ccbd-cdf5-4a53-b529-dc91c26cad1d service cinder] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:59.968676 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67275e8a-e59a-4b88-9a66-e44f3fe0d7f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:59.969333 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67275e8a-e59a-4b88-9a66-e44f3fe0d7f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:59.969613 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67275e8a-e59a-4b88-9a66-e44f3fe0d7f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:59.969887 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67275e8a-e59a-4b88-9a66-e44f3fe0d7f2 admin admin] PATH_INFO: `/v3/users/e97dd542122e41eaa7618247573433aa` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:59.971052 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67275e8a-e59a-4b88-9a66-e44f3fe0d7f2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e97dd542122e41eaa7618247573433aa)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:59.973012 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67275e8a-e59a-4b88-9a66-e44f3fe0d7f2 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:52:00.142480 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=90,keystone:INSERT=5,keystone:DELETE=5 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:52:00.152776 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-67275e8a-e59a-4b88-9a66-e44f3fe0d7f2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e97dd542122e41eaa7618247573433aa'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:52:00.159746 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-67275e8a-e59a-4b88-9a66-e44f3fe0d7f2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e97dd542122e41eaa7618247573433aa'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:52:00.160478 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-67275e8a-e59a-4b88-9a66-e44f3fe0d7f2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e97dd542122e41eaa7618247573433aa'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:52:00.161114 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-67275e8a-e59a-4b88-9a66-e44f3fe0d7f2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e97dd542122e41eaa7618247573433aa'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:52:00.248367 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-67275e8a-e59a-4b88-9a66-e44f3fe0d7f2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e97dd542122e41eaa7618247573433aa'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:52:00.271139 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1262/2523] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Aug 24 06:51:59 2022] DELETE /identity/v3/users/e97dd542122e41eaa7618247573433aa => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:52:00.275747 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22562667-ce94-493e-a3ce-d14a279e0167 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:00.276348 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22562667-ce94-493e-a3ce-d14a279e0167 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:00.276348 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22562667-ce94-493e-a3ce-d14a279e0167 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:00.297742 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-22562667-ce94-493e-a3ce-d14a279e0167 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:52:00.299014 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-22562667-ce94-493e-a3ce-d14a279e0167 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:52:00.369768 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1262/2524] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:52:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:52:00.383007 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67275e8a-e59a-4b88-9a66-e44f3fe0d7f2 admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:52:00.384590 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-67275e8a-e59a-4b88-9a66-e44f3fe0d7f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:52:00.444693 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67275e8a-e59a-4b88-9a66-e44f3fe0d7f2 admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:52:00.448422 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395542ef-98d3-498f-97d9-f5ed59b485c3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bb6c79db75e4d169331c02133d1ec32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:52:00.449637 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395542ef-98d3-498f-97d9-f5ed59b485c3 service neutron] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:00.449907 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395542ef-98d3-498f-97d9-f5ed59b485c3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:00.450177 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395542ef-98d3-498f-97d9-f5ed59b485c3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:00.460755 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395542ef-98d3-498f-97d9-f5ed59b485c3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:52:00.462464 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395542ef-98d3-498f-97d9-f5ed59b485c3 service neutron] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:52:00.473319 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1263/2525] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:52:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:52:00.989361 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22562667-ce94-493e-a3ce-d14a279e0167 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:52:00.991213 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-22562667-ce94-493e-a3ce-d14a279e0167 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:52:01.013039 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22562667-ce94-493e-a3ce-d14a279e0167 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:52:01.022671 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cebb9c-0bc5-414f-a2c7-5c1ff8976fb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:52:01.023494 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cebb9c-0bc5-414f-a2c7-5c1ff8976fb1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:01.023828 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cebb9c-0bc5-414f-a2c7-5c1ff8976fb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:01.024193 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cebb9c-0bc5-414f-a2c7-5c1ff8976fb1 admin admin] PATH_INFO: `/v3/projects/834d5c0f457b4a24b72bb098ec6193ae` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:01.025470 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cebb9c-0bc5-414f-a2c7-5c1ff8976fb1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=834d5c0f457b4a24b72bb098ec6193ae)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:52:01.028001 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cebb9c-0bc5-414f-a2c7-5c1ff8976fb1 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:52:01.131860 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-b5cebb9c-0bc5-414f-a2c7-5c1ff8976fb1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '834d5c0f457b4a24b72bb098ec6193ae', 'request_id': 'req-b5cebb9c-0bc5-414f-a2c7-5c1ff8976fb1'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:52:01.138314 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-b5cebb9c-0bc5-414f-a2c7-5c1ff8976fb1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '834d5c0f457b4a24b72bb098ec6193ae', 'request_id': 'req-b5cebb9c-0bc5-414f-a2c7-5c1ff8976fb1'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:52:01.145130 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-b5cebb9c-0bc5-414f-a2c7-5c1ff8976fb1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '834d5c0f457b4a24b72bb098ec6193ae', 'request_id': 'req-b5cebb9c-0bc5-414f-a2c7-5c1ff8976fb1'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:52:01.265954 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-b5cebb9c-0bc5-414f-a2c7-5c1ff8976fb1 admin admin] The token cache is being invalidate because project 834d5c0f457b4a24b72bb098ec6193ae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81852) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:52:01.266255 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-b5cebb9c-0bc5-414f-a2c7-5c1ff8976fb1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:52:01.266848 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-b5cebb9c-0bc5-414f-a2c7-5c1ff8976fb1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:52:01.268379 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1263/2526] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Aug 24 06:52:00 2022] DELETE /identity/v3/projects/834d5c0f457b4a24b72bb098ec6193ae => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:52:01.272475 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395542ef-98d3-498f-97d9-f5ed59b485c3 service neutron] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:52:01.283601 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395542ef-98d3-498f-97d9-f5ed59b485c3 service neutron] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:52:01.352313 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18137ef2-deb1-479a-913b-9bb2642451e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:52:01.352968 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18137ef2-deb1-479a-913b-9bb2642451e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:01.353253 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18137ef2-deb1-479a-913b-9bb2642451e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:01.353594 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18137ef2-deb1-479a-913b-9bb2642451e8 admin admin] PATH_INFO: `/v3/users/6e2b58f1c2f1437089ba9b59d53e7c7f` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:01.354938 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18137ef2-deb1-479a-913b-9bb2642451e8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6e2b58f1c2f1437089ba9b59d53e7c7f)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:52:01.357651 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18137ef2-deb1-479a-913b-9bb2642451e8 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:52:01.561280 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cebb9c-0bc5-414f-a2c7-5c1ff8976fb1 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:52:01.562903 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-b5cebb9c-0bc5-414f-a2c7-5c1ff8976fb1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:52:01.577082 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cebb9c-0bc5-414f-a2c7-5c1ff8976fb1 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:52:01.587793 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c1ea46-a21b-4012-adcf-1196330d929c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:52:01.588439 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c1ea46-a21b-4012-adcf-1196330d929c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:01.588757 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c1ea46-a21b-4012-adcf-1196330d929c admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:01.589028 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c1ea46-a21b-4012-adcf-1196330d929c admin admin] PATH_INFO: `/v3/users/cbc0eae15e4044c498fdd8412e0f038d` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:01.590775 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c1ea46-a21b-4012-adcf-1196330d929c admin admin] RBAC: Authorizing `identity:delete_user(user_id=cbc0eae15e4044c498fdd8412e0f038d)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:52:01.592766 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c1ea46-a21b-4012-adcf-1196330d929c admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:52:01.649606 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-18137ef2-deb1-479a-913b-9bb2642451e8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6e2b58f1c2f1437089ba9b59d53e7c7f'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:52:01.658185 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-18137ef2-deb1-479a-913b-9bb2642451e8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6e2b58f1c2f1437089ba9b59d53e7c7f'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:52:01.658865 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-18137ef2-deb1-479a-913b-9bb2642451e8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6e2b58f1c2f1437089ba9b59d53e7c7f'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:52:01.659492 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-18137ef2-deb1-479a-913b-9bb2642451e8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6e2b58f1c2f1437089ba9b59d53e7c7f'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:52:01.661492 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:52:01.763877 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-18137ef2-deb1-479a-913b-9bb2642451e8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6e2b58f1c2f1437089ba9b59d53e7c7f'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:52:01.783598 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1264/2527] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Aug 24 06:52:01 2022] DELETE /identity/v3/users/6e2b58f1c2f1437089ba9b59d53e7c7f => generated 0 bytes in 512 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:52:01.804292 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:52:01.820666 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-d6c1ea46-a21b-4012-adcf-1196330d929c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cbc0eae15e4044c498fdd8412e0f038d'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:52:01.829185 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-d6c1ea46-a21b-4012-adcf-1196330d929c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cbc0eae15e4044c498fdd8412e0f038d'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:52:01.832645 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-d6c1ea46-a21b-4012-adcf-1196330d929c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cbc0eae15e4044c498fdd8412e0f038d'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:52:01.905589 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-d6c1ea46-a21b-4012-adcf-1196330d929c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cbc0eae15e4044c498fdd8412e0f038d'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:52:01.918570 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-d6c1ea46-a21b-4012-adcf-1196330d929c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cbc0eae15e4044c498fdd8412e0f038d'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:52:01.924796 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1264/2528] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Aug 24 06:52:01 2022] DELETE /identity/v3/users/cbc0eae15e4044c498fdd8412e0f038d => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:52:01.929398 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a50c7e5-b3ee-406b-b5ad-5cfcb57bba11 None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:01.929688 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a50c7e5-b3ee-406b-b5ad-5cfcb57bba11 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:01.929967 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a50c7e5-b3ee-406b-b5ad-5cfcb57bba11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:01.956558 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-0a50c7e5-b3ee-406b-b5ad-5cfcb57bba11 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:52:01.957807 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-0a50c7e5-b3ee-406b-b5ad-5cfcb57bba11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:52:02.040544 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1265/2529] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:52:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:52:02.050696 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c1ea46-a21b-4012-adcf-1196330d929c admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:52:02.052282 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-d6c1ea46-a21b-4012-adcf-1196330d929c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:52:02.119768 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c1ea46-a21b-4012-adcf-1196330d929c admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:52:02.124347 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcef89ec-6be9-40a7-a3cd-f0b6d6c1f14a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bb6c79db75e4d169331c02133d1ec32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:52:02.124991 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcef89ec-6be9-40a7-a3cd-f0b6d6c1f14a service neutron] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:02.125270 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcef89ec-6be9-40a7-a3cd-f0b6d6c1f14a service neutron] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:02.125550 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcef89ec-6be9-40a7-a3cd-f0b6d6c1f14a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:02.136484 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcef89ec-6be9-40a7-a3cd-f0b6d6c1f14a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:52:02.138009 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcef89ec-6be9-40a7-a3cd-f0b6d6c1f14a service neutron] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:52:02.150213 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1265/2530] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:52:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:52:02.276152 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a50c7e5-b3ee-406b-b5ad-5cfcb57bba11 None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:52:02.277782 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-0a50c7e5-b3ee-406b-b5ad-5cfcb57bba11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:52:02.303802 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a50c7e5-b3ee-406b-b5ad-5cfcb57bba11 None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:52:02.315484 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb4c5ba-808b-4116-b56a-fb7edec59b5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:52:02.316367 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb4c5ba-808b-4116-b56a-fb7edec59b5c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:02.316873 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb4c5ba-808b-4116-b56a-fb7edec59b5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:02.317250 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb4c5ba-808b-4116-b56a-fb7edec59b5c admin admin] PATH_INFO: `/v3/projects/ee04558220be4e3a9ecc92ea6d3f33c7` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:02.318938 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb4c5ba-808b-4116-b56a-fb7edec59b5c admin admin] RBAC: Authorizing `identity:delete_project(project_id=ee04558220be4e3a9ecc92ea6d3f33c7)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:52:02.321366 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb4c5ba-808b-4116-b56a-fb7edec59b5c admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:52:02.474277 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-1bb4c5ba-808b-4116-b56a-fb7edec59b5c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ee04558220be4e3a9ecc92ea6d3f33c7', 'request_id': 'req-1bb4c5ba-808b-4116-b56a-fb7edec59b5c'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:52:02.482117 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-1bb4c5ba-808b-4116-b56a-fb7edec59b5c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ee04558220be4e3a9ecc92ea6d3f33c7', 'request_id': 'req-1bb4c5ba-808b-4116-b56a-fb7edec59b5c'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:52:02.491808 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-1bb4c5ba-808b-4116-b56a-fb7edec59b5c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ee04558220be4e3a9ecc92ea6d3f33c7', 'request_id': 'req-1bb4c5ba-808b-4116-b56a-fb7edec59b5c'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:52:02.574684 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-1bb4c5ba-808b-4116-b56a-fb7edec59b5c admin admin] The token cache is being invalidate because project ee04558220be4e3a9ecc92ea6d3f33c7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81851) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:52:02.576523 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-1bb4c5ba-808b-4116-b56a-fb7edec59b5c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:52:02.576523 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-1bb4c5ba-808b-4116-b56a-fb7edec59b5c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:52:02.577980 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1266/2531] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Aug 24 06:52:02 2022] DELETE /identity/v3/projects/ee04558220be4e3a9ecc92ea6d3f33c7 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:52:02.583587 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcef89ec-6be9-40a7-a3cd-f0b6d6c1f14a service neutron] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:52:02.592530 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcef89ec-6be9-40a7-a3cd-f0b6d6c1f14a service neutron] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:52:02.666578 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bec14b1-1290-4ef3-a0bf-f715f74d545b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:52:02.667228 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bec14b1-1290-4ef3-a0bf-f715f74d545b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:02.667550 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bec14b1-1290-4ef3-a0bf-f715f74d545b admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:02.667955 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bec14b1-1290-4ef3-a0bf-f715f74d545b admin admin] PATH_INFO: `/v3/users/83f575b5ac08431eb5d0f0ed9d2250f7` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:02.669104 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bec14b1-1290-4ef3-a0bf-f715f74d545b admin admin] RBAC: Authorizing `identity:delete_user(user_id=83f575b5ac08431eb5d0f0ed9d2250f7)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:52:02.671078 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bec14b1-1290-4ef3-a0bf-f715f74d545b admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:52:02.730282 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb4c5ba-808b-4116-b56a-fb7edec59b5c admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:52:02.731876 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-1bb4c5ba-808b-4116-b56a-fb7edec59b5c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:52:02.750205 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb4c5ba-808b-4116-b56a-fb7edec59b5c admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:52:02.762006 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33645b2-1f53-4477-a298-01898a2bf6c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:52:02.762006 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33645b2-1f53-4477-a298-01898a2bf6c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:02.762006 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33645b2-1f53-4477-a298-01898a2bf6c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:02.762006 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33645b2-1f53-4477-a298-01898a2bf6c7 admin admin] PATH_INFO: `/v3/projects/d2aa8824715e47ab9ef1d8b459050c7f` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:02.762428 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33645b2-1f53-4477-a298-01898a2bf6c7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d2aa8824715e47ab9ef1d8b459050c7f)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:52:02.766420 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33645b2-1f53-4477-a298-01898a2bf6c7 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:52:02.892752 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-9bec14b1-1290-4ef3-a0bf-f715f74d545b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '83f575b5ac08431eb5d0f0ed9d2250f7'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:52:02.897383 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:52:02.900368 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-9bec14b1-1290-4ef3-a0bf-f715f74d545b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '83f575b5ac08431eb5d0f0ed9d2250f7'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:52:02.901129 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-9bec14b1-1290-4ef3-a0bf-f715f74d545b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '83f575b5ac08431eb5d0f0ed9d2250f7'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:52:02.941461 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-e33645b2-1f53-4477-a298-01898a2bf6c7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd2aa8824715e47ab9ef1d8b459050c7f', 'request_id': 'req-e33645b2-1f53-4477-a298-01898a2bf6c7'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:52:02.943680 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:INSERT=2,keystone:DELETE=15,keystone:SELECT=83 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:52:02.947593 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-e33645b2-1f53-4477-a298-01898a2bf6c7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd2aa8824715e47ab9ef1d8b459050c7f', 'request_id': 'req-e33645b2-1f53-4477-a298-01898a2bf6c7'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:52:02.954804 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-e33645b2-1f53-4477-a298-01898a2bf6c7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd2aa8824715e47ab9ef1d8b459050c7f', 'request_id': 'req-e33645b2-1f53-4477-a298-01898a2bf6c7'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:52:02.977182 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-9bec14b1-1290-4ef3-a0bf-f715f74d545b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '83f575b5ac08431eb5d0f0ed9d2250f7'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:52:02.991286 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-9bec14b1-1290-4ef3-a0bf-f715f74d545b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '83f575b5ac08431eb5d0f0ed9d2250f7'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:52:02.998022 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1266/2532] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Aug 24 06:52:02 2022] DELETE /identity/v3/users/83f575b5ac08431eb5d0f0ed9d2250f7 => generated 0 bytes in 416 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 06:52:03.070498 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-e33645b2-1f53-4477-a298-01898a2bf6c7 admin admin] The token cache is being invalidate because project d2aa8824715e47ab9ef1d8b459050c7f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81851) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:52:03.070825 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-e33645b2-1f53-4477-a298-01898a2bf6c7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:52:03.071552 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-e33645b2-1f53-4477-a298-01898a2bf6c7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:52:03.073353 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1267/2533] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Aug 24 06:52:02 2022] DELETE /identity/v3/projects/d2aa8824715e47ab9ef1d8b459050c7f => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:52:03.079422 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad46a4b-c208-40c9-9820-c9f9d58bb968 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:03.079760 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad46a4b-c208-40c9-9820-c9f9d58bb968 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:03.079974 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad46a4b-c208-40c9-9820-c9f9d58bb968 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:03.101841 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-0ad46a4b-c208-40c9-9820-c9f9d58bb968 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:52:03.103077 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-0ad46a4b-c208-40c9-9820-c9f9d58bb968 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:52:03.168352 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1267/2534] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:52:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:52:03.176171 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f5499a-eab6-4beb-93d8-cb8f3cd77b36 None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:03.176455 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f5499a-eab6-4beb-93d8-cb8f3cd77b36 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:03.176765 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f5499a-eab6-4beb-93d8-cb8f3cd77b36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:03.200810 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-74f5499a-eab6-4beb-93d8-cb8f3cd77b36 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:52:03.202016 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-74f5499a-eab6-4beb-93d8-cb8f3cd77b36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:52:03.211614 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1268/2535] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:52:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:52:03.215608 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad46a4b-c208-40c9-9820-c9f9d58bb968 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:52:03.225300 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad46a4b-c208-40c9-9820-c9f9d58bb968 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:52:03.230673 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe88354e-405f-411b-978d-1874dd842b85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:52:03.231311 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe88354e-405f-411b-978d-1874dd842b85 admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:03.231586 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe88354e-405f-411b-978d-1874dd842b85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:03.231857 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe88354e-405f-411b-978d-1874dd842b85 admin admin] PATH_INFO: `/v3/domains` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:03.235330 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe88354e-405f-411b-978d-1874dd842b85 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:52:03.235330 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe88354e-405f-411b-978d-1874dd842b85 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:52:03.235330 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe88354e-405f-411b-978d-1874dd842b85 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:52:03.250762 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1268/2536] 10.222.0.62 () {58 vars in 1153 bytes} [Wed Aug 24 06:52:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:52:03.253759 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f5499a-eab6-4beb-93d8-cb8f3cd77b36 None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:52:03.264151 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f5499a-eab6-4beb-93d8-cb8f3cd77b36 None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:52:03.270000 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d6d4af-2260-47b6-9c74-4eaa3848388a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:52:03.270944 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d6d4af-2260-47b6-9c74-4eaa3848388a admin admin] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:03.271394 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d6d4af-2260-47b6-9c74-4eaa3848388a admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:03.271805 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d6d4af-2260-47b6-9c74-4eaa3848388a admin admin] PATH_INFO: `/v3/projects` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:03.272581 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d6d4af-2260-47b6-9c74-4eaa3848388a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:52:03.275170 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d6d4af-2260-47b6-9c74-4eaa3848388a admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:52:03.376992 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1269/2537] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Aug 24 06:52:03 2022] POST /identity/v3/projects => generated 395 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:52:03.381042 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe88354e-405f-411b-978d-1874dd842b85 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:52:03.391210 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe88354e-405f-411b-978d-1874dd842b85 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:52:03.449475 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b3c499-ff8b-4c17-a1ca-58c84efa4bca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:52:03.450046 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b3c499-ff8b-4c17-a1ca-58c84efa4bca admin admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:03.450324 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b3c499-ff8b-4c17-a1ca-58c84efa4bca admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:03.450595 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b3c499-ff8b-4c17-a1ca-58c84efa4bca admin admin] PATH_INFO: `/v3/users` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:03.451221 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b3c499-ff8b-4c17-a1ca-58c84efa4bca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:52:03.453208 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b3c499-ff8b-4c17-a1ca-58c84efa4bca admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:52:03.462293 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d6d4af-2260-47b6-9c74-4eaa3848388a admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:52:03.466397 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-54d6d4af-2260-47b6-9c74-4eaa3848388a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:52:03.481954 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d6d4af-2260-47b6-9c74-4eaa3848388a admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:52:03.488250 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3694c7-eb44-4ba0-9059-478afc40c586 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:52:03.488969 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3694c7-eb44-4ba0-9059-478afc40c586 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:03.489250 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3694c7-eb44-4ba0-9059-478afc40c586 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:03.489520 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3694c7-eb44-4ba0-9059-478afc40c586 admin admin] PATH_INFO: `/v3/projects/b04308fbe979463fa9014d1f50051687` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:03.490768 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3694c7-eb44-4ba0-9059-478afc40c586 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b04308fbe979463fa9014d1f50051687)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:52:03.492889 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3694c7-eb44-4ba0-9059-478afc40c586 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:52:03.567874 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1269/2538] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Aug 24 06:52:03 2022] POST /identity/v3/users => generated 298 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:52:03.569068 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b3c499-ff8b-4c17-a1ca-58c84efa4bca admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:52:03.579870 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b3c499-ff8b-4c17-a1ca-58c84efa4bca admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:52:03.584739 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bec3d6-c5bb-4d48-ba5c-7118b36d367b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:52:03.585357 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bec3d6-c5bb-4d48-ba5c-7118b36d367b admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:03.585637 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bec3d6-c5bb-4d48-ba5c-7118b36d367b admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:03.585911 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bec3d6-c5bb-4d48-ba5c-7118b36d367b admin admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:03.586582 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52bec3d6-c5bb-4d48-ba5c-7118b36d367b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:52:03.588216 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52bec3d6-c5bb-4d48-ba5c-7118b36d367b admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:52:03.600168 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1270/2539] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Aug 24 06:52:03 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 06:52:03.603946 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bec3d6-c5bb-4d48-ba5c-7118b36d367b admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:52:03.615011 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bec3d6-c5bb-4d48-ba5c-7118b36d367b admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:52:03.623838 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-3c3694c7-eb44-4ba0-9059-478afc40c586 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b04308fbe979463fa9014d1f50051687', 'request_id': 'req-3c3694c7-eb44-4ba0-9059-478afc40c586'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:52:03.630205 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-3c3694c7-eb44-4ba0-9059-478afc40c586 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b04308fbe979463fa9014d1f50051687', 'request_id': 'req-3c3694c7-eb44-4ba0-9059-478afc40c586'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:52:03.637751 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-3c3694c7-eb44-4ba0-9059-478afc40c586 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b04308fbe979463fa9014d1f50051687', 'request_id': 'req-3c3694c7-eb44-4ba0-9059-478afc40c586'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:52:03.649828 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:52:03.678968 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0708fb84-a6b5-46ab-bc5f-422981069b8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:52:03.679609 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0708fb84-a6b5-46ab-bc5f-422981069b8a admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:03.679902 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0708fb84-a6b5-46ab-bc5f-422981069b8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:03.680198 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0708fb84-a6b5-46ab-bc5f-422981069b8a admin admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:03.680875 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0708fb84-a6b5-46ab-bc5f-422981069b8a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:52:03.682628 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0708fb84-a6b5-46ab-bc5f-422981069b8a admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:52:03.695990 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1271/2540] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Aug 24 06:52:03 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 06:52:03.700793 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0708fb84-a6b5-46ab-bc5f-422981069b8a admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:52:03.711792 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0708fb84-a6b5-46ab-bc5f-422981069b8a admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:52:03.717317 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eeb78ba-d718-4a14-8281-ad38d4482e4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:52:03.718167 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eeb78ba-d718-4a14-8281-ad38d4482e4f admin admin] REQUEST_METHOD: `PUT` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:03.718506 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eeb78ba-d718-4a14-8281-ad38d4482e4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:03.718707 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eeb78ba-d718-4a14-8281-ad38d4482e4f admin admin] PATH_INFO: `/v3/projects/6118a695540f486cbad61fe0d566e457/users/b9528268491641fcb78f37c2bb35e328/roles/580aa7dbb38d4cff8f9d1bb8983f99ef` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:03.734902 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eeb78ba-d718-4a14-8281-ad38d4482e4f admin admin] RBAC: Authorizing `identity:create_grant(project_id=6118a695540f486cbad61fe0d566e457, user_id=b9528268491641fcb78f37c2bb35e328, role_id=580aa7dbb38d4cff8f9d1bb8983f99ef)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:52:03.737720 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eeb78ba-d718-4a14-8281-ad38d4482e4f admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:52:03.753478 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-3c3694c7-eb44-4ba0-9059-478afc40c586 admin admin] The token cache is being invalidate because project b04308fbe979463fa9014d1f50051687 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81851) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:52:03.753823 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-3c3694c7-eb44-4ba0-9059-478afc40c586 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:52:03.754467 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-3c3694c7-eb44-4ba0-9059-478afc40c586 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:52:03.755986 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1270/2541] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Aug 24 06:52:03 2022] DELETE /identity/v3/projects/b04308fbe979463fa9014d1f50051687 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:52:03.762343 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cacfd3d-41dd-4943-9252-8aaa953fa366 None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:03.762627 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cacfd3d-41dd-4943-9252-8aaa953fa366 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:03.762896 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cacfd3d-41dd-4943-9252-8aaa953fa366 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:03.787714 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-7cacfd3d-41dd-4943-9252-8aaa953fa366 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:52:03.789097 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-7cacfd3d-41dd-4943-9252-8aaa953fa366 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:52:03.801841 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1271/2542] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:52:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 06:52:03.804906 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1272/2543] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Aug 24 06:52:03 2022] PUT /identity/v3/projects/6118a695540f486cbad61fe0d566e457/users/b9528268491641fcb78f37c2bb35e328/roles/580aa7dbb38d4cff8f9d1bb8983f99ef => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 06:52:03.813922 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38702cb5-8020-4729-812f-0ff15a977d8e None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:03.813922 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38702cb5-8020-4729-812f-0ff15a977d8e None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:03.813922 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38702cb5-8020-4729-812f-0ff15a977d8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:03.818375 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0b8cf2-f9c5-4409-8f2e-584d1e2cc476 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:03.818671 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0b8cf2-f9c5-4409-8f2e-584d1e2cc476 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:03.818949 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0b8cf2-f9c5-4409-8f2e-584d1e2cc476 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:03.834187 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-38702cb5-8020-4729-812f-0ff15a977d8e None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:52:03.837247 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-38702cb5-8020-4729-812f-0ff15a977d8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:52:03.852541 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-ed0b8cf2-f9c5-4409-8f2e-584d1e2cc476 None None] MFA Rules not processed for user `b9528268491641fcb78f37c2bb35e328`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:52:03.854231 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-ed0b8cf2-f9c5-4409-8f2e-584d1e2cc476 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:52:03.921630 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1272/2544] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:52:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:52:03.922685 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38702cb5-8020-4729-812f-0ff15a977d8e None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:52:03.923762 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1273/2545] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:52:03 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:52:03.935560 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38702cb5-8020-4729-812f-0ff15a977d8e None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:52:03.935732 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0b8cf2-f9c5-4409-8f2e-584d1e2cc476 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:52:03.937132 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-ed0b8cf2-f9c5-4409-8f2e-584d1e2cc476 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:52:03.940232 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e193166-6e4a-4378-bae1-c571f59627b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:52:03.940878 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e193166-6e4a-4378-bae1-c571f59627b9 admin admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:03.941150 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e193166-6e4a-4378-bae1-c571f59627b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:03.941446 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e193166-6e4a-4378-bae1-c571f59627b9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:03.942093 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e193166-6e4a-4378-bae1-c571f59627b9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:52:03.942391 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e193166-6e4a-4378-bae1-c571f59627b9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:52:03.944028 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e193166-6e4a-4378-bae1-c571f59627b9 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:52:03.963505 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1273/2546] 10.222.0.62 () {58 vars in 1153 bytes} [Wed Aug 24 06:52:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:52:03.965095 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e193166-6e4a-4378-bae1-c571f59627b9 admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:52:03.975043 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e193166-6e4a-4378-bae1-c571f59627b9 admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:52:03.982901 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72935a0-1e9f-428e-8ebf-b820079447be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:52:03.983643 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72935a0-1e9f-428e-8ebf-b820079447be admin admin] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:03.983782 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72935a0-1e9f-428e-8ebf-b820079447be admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:03.984048 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72935a0-1e9f-428e-8ebf-b820079447be admin admin] PATH_INFO: `/v3/projects` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:03.984740 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72935a0-1e9f-428e-8ebf-b820079447be admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:52:03.986661 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72935a0-1e9f-428e-8ebf-b820079447be admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:52:04.001200 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0b8cf2-f9c5-4409-8f2e-584d1e2cc476 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:52:04.005756 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98381244-183a-4007-bc2e-5d45c4f2996b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c4def55ac0e48b29836bc5c64bc7402', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:52:04.006377 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98381244-183a-4007-bc2e-5d45c4f2996b service cinder] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:04.006484 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=88,keystone:DELETE=9,keystone:INSERT=3 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:52:04.006653 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98381244-183a-4007-bc2e-5d45c4f2996b service cinder] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:04.006926 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98381244-183a-4007-bc2e-5d45c4f2996b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:04.016559 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98381244-183a-4007-bc2e-5d45c4f2996b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:52:04.018009 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98381244-183a-4007-bc2e-5d45c4f2996b service cinder] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:52:04.028103 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1274/2547] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:52:03 2022] GET /identity/v3/auth/tokens => generated 3272 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 06:52:04.109035 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1274/2548] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Aug 24 06:52:03 2022] POST /identity/v3/projects => generated 379 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 06:52:04.115255 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98381244-183a-4007-bc2e-5d45c4f2996b service cinder] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:52:04.127910 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98381244-183a-4007-bc2e-5d45c4f2996b service cinder] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:52:04.177064 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:52:04.196998 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085ba667-8216-4e65-9e6f-b9e44d1e72bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:52:04.197611 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085ba667-8216-4e65-9e6f-b9e44d1e72bf admin admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:04.197886 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085ba667-8216-4e65-9e6f-b9e44d1e72bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:04.198158 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085ba667-8216-4e65-9e6f-b9e44d1e72bf admin admin] PATH_INFO: `/v3/users` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:04.199300 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085ba667-8216-4e65-9e6f-b9e44d1e72bf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:52:04.201257 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085ba667-8216-4e65-9e6f-b9e44d1e72bf admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:52:04.281078 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1275/2549] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Aug 24 06:52:04 2022] POST /identity/v3/users => generated 290 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:52:04.282811 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72935a0-1e9f-428e-8ebf-b820079447be admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:52:04.295374 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72935a0-1e9f-428e-8ebf-b820079447be admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:52:04.301713 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814bf37d-a18b-47e1-a157-7a338340d6ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:52:04.302428 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814bf37d-a18b-47e1-a157-7a338340d6ff admin admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:04.302800 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814bf37d-a18b-47e1-a157-7a338340d6ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:04.303215 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814bf37d-a18b-47e1-a157-7a338340d6ff admin admin] PATH_INFO: `/v3/roles` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:04.304278 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814bf37d-a18b-47e1-a157-7a338340d6ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:52:04.306023 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814bf37d-a18b-47e1-a157-7a338340d6ff admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:52:04.317803 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1275/2550] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Aug 24 06:52:04 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:52:04.321937 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085ba667-8216-4e65-9e6f-b9e44d1e72bf admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:52:04.332963 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085ba667-8216-4e65-9e6f-b9e44d1e72bf admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:52:04.337699 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ebb8f5-d73f-4f4d-98e8-77ae0542d02f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:52:04.338292 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ebb8f5-d73f-4f4d-98e8-77ae0542d02f admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:04.338564 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ebb8f5-d73f-4f4d-98e8-77ae0542d02f admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:04.338833 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ebb8f5-d73f-4f4d-98e8-77ae0542d02f admin admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:04.339465 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ebb8f5-d73f-4f4d-98e8-77ae0542d02f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:52:04.341158 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ebb8f5-d73f-4f4d-98e8-77ae0542d02f admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:52:04.352185 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1276/2551] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Aug 24 06:52:04 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:52:04.356394 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814bf37d-a18b-47e1-a157-7a338340d6ff admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:52:04.366621 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814bf37d-a18b-47e1-a157-7a338340d6ff admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:52:04.372109 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6382d4b1-c65d-4768-aef4-cf0bdd830560 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:52:04.372908 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6382d4b1-c65d-4768-aef4-cf0bdd830560 admin admin] REQUEST_METHOD: `PUT` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:04.373187 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6382d4b1-c65d-4768-aef4-cf0bdd830560 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:04.373457 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6382d4b1-c65d-4768-aef4-cf0bdd830560 admin admin] PATH_INFO: `/v3/projects/f95c9b09fc2f47baa1d8c0b08a54e7e2/users/a5943c9c24024f67b78f1db5241f1ef5/roles/580aa7dbb38d4cff8f9d1bb8983f99ef` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:04.389941 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6382d4b1-c65d-4768-aef4-cf0bdd830560 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f95c9b09fc2f47baa1d8c0b08a54e7e2, user_id=a5943c9c24024f67b78f1db5241f1ef5, role_id=580aa7dbb38d4cff8f9d1bb8983f99ef)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:52:04.392639 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6382d4b1-c65d-4768-aef4-cf0bdd830560 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:52:04.483672 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1276/2552] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Aug 24 06:52:04 2022] PUT /identity/v3/projects/f95c9b09fc2f47baa1d8c0b08a54e7e2/users/a5943c9c24024f67b78f1db5241f1ef5/roles/580aa7dbb38d4cff8f9d1bb8983f99ef => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:52:04.495042 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e05ecce-2c18-44ac-ad4a-aab3ffee70c1 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:04.495329 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e05ecce-2c18-44ac-ad4a-aab3ffee70c1 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:04.496148 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e05ecce-2c18-44ac-ad4a-aab3ffee70c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:04.525102 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-0e05ecce-2c18-44ac-ad4a-aab3ffee70c1 None None] MFA Rules not processed for user `a5943c9c24024f67b78f1db5241f1ef5`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:52:04.526332 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-0e05ecce-2c18-44ac-ad4a-aab3ffee70c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:52:04.599327 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1277/2553] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:52:04 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:52:04.604192 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6382d4b1-c65d-4768-aef4-cf0bdd830560 admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:52:04.689198 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6382d4b1-c65d-4768-aef4-cf0bdd830560 admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:52:04.694332 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2016a2e-3d82-4004-a2f8-b19f991edf1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:52:04.694943 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2016a2e-3d82-4004-a2f8-b19f991edf1c admin admin] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:04.695220 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2016a2e-3d82-4004-a2f8-b19f991edf1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:04.695491 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2016a2e-3d82-4004-a2f8-b19f991edf1c admin admin] PATH_INFO: `/v3/projects` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:04.696144 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2016a2e-3d82-4004-a2f8-b19f991edf1c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:52:04.698175 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2016a2e-3d82-4004-a2f8-b19f991edf1c admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:52:04.837189 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1277/2554] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Aug 24 06:52:04 2022] POST /identity/v3/projects => generated 381 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:52:04.843029 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e05ecce-2c18-44ac-ad4a-aab3ffee70c1 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:52:04.856161 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e05ecce-2c18-44ac-ad4a-aab3ffee70c1 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:52:04.966826 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab49f3c2-51e2-40a6-bce0-c81bc44aa93f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:52:04.967703 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab49f3c2-51e2-40a6-bce0-c81bc44aa93f admin admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:04.968040 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab49f3c2-51e2-40a6-bce0-c81bc44aa93f admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:04.968481 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab49f3c2-51e2-40a6-bce0-c81bc44aa93f admin admin] PATH_INFO: `/v3/users` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:04.969215 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab49f3c2-51e2-40a6-bce0-c81bc44aa93f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:52:04.971199 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab49f3c2-51e2-40a6-bce0-c81bc44aa93f admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:52:05.060392 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1278/2555] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Aug 24 06:52:04 2022] POST /identity/v3/users => generated 297 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 06:52:05.064828 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2016a2e-3d82-4004-a2f8-b19f991edf1c admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:52:05.075301 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2016a2e-3d82-4004-a2f8-b19f991edf1c admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:52:05.079938 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c35173c-4cc8-45fb-8505-3e53a929d368 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:52:05.080599 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c35173c-4cc8-45fb-8505-3e53a929d368 admin admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:05.080885 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c35173c-4cc8-45fb-8505-3e53a929d368 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:05.081158 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c35173c-4cc8-45fb-8505-3e53a929d368 admin admin] PATH_INFO: `/v3/roles` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:05.081803 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c35173c-4cc8-45fb-8505-3e53a929d368 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:52:05.083402 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c35173c-4cc8-45fb-8505-3e53a929d368 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:52:05.095833 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1278/2556] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Aug 24 06:52:05 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 06:52:05.100119 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab49f3c2-51e2-40a6-bce0-c81bc44aa93f admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:52:05.110815 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab49f3c2-51e2-40a6-bce0-c81bc44aa93f admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:52:05.115452 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d822bf-3585-4d40-88fb-862902799587 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:52:05.116112 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d822bf-3585-4d40-88fb-862902799587 admin admin] REQUEST_METHOD: `PUT` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:05.116392 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d822bf-3585-4d40-88fb-862902799587 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:05.116779 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d822bf-3585-4d40-88fb-862902799587 admin admin] PATH_INFO: `/v3/projects/528a8395e2d442ffae72fcc9c53b907d/users/04e442bac7b44c819467fca141c0220b/roles/49e47cd157d743f6905f1db07ec878a7` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:05.131610 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d822bf-3585-4d40-88fb-862902799587 admin admin] RBAC: Authorizing `identity:create_grant(project_id=528a8395e2d442ffae72fcc9c53b907d, user_id=04e442bac7b44c819467fca141c0220b, role_id=49e47cd157d743f6905f1db07ec878a7)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:52:05.134513 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d822bf-3585-4d40-88fb-862902799587 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:52:05.208976 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1279/2557] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Aug 24 06:52:05 2022] PUT /identity/v3/projects/528a8395e2d442ffae72fcc9c53b907d/users/04e442bac7b44c819467fca141c0220b/roles/49e47cd157d743f6905f1db07ec878a7 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:52:05.213096 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c35173c-4cc8-45fb-8505-3e53a929d368 admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:52:05.223767 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c35173c-4cc8-45fb-8505-3e53a929d368 admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:52:05.284199 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2618c046-1ab7-4632-b4c3-2cd8b7f3b317 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:52:05.284761 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2618c046-1ab7-4632-b4c3-2cd8b7f3b317 admin admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:05.285039 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2618c046-1ab7-4632-b4c3-2cd8b7f3b317 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:05.285310 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2618c046-1ab7-4632-b4c3-2cd8b7f3b317 admin admin] PATH_INFO: `/v3/roles` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:05.285940 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2618c046-1ab7-4632-b4c3-2cd8b7f3b317 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:52:05.287575 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2618c046-1ab7-4632-b4c3-2cd8b7f3b317 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:52:05.297991 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1279/2558] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Aug 24 06:52:05 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:52:05.302166 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d822bf-3585-4d40-88fb-862902799587 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:52:05.313418 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d822bf-3585-4d40-88fb-862902799587 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:52:05.318074 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a70c9d-6392-4e9a-91ab-e863610fc87f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:52:05.318721 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a70c9d-6392-4e9a-91ab-e863610fc87f admin admin] REQUEST_METHOD: `PUT` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:05.318999 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a70c9d-6392-4e9a-91ab-e863610fc87f admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:05.319294 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a70c9d-6392-4e9a-91ab-e863610fc87f admin admin] PATH_INFO: `/v3/projects/528a8395e2d442ffae72fcc9c53b907d/users/04e442bac7b44c819467fca141c0220b/roles/49e47cd157d743f6905f1db07ec878a7` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:05.321253 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a70c9d-6392-4e9a-91ab-e863610fc87f admin admin] RBAC: Authorizing `identity:create_grant(project_id=528a8395e2d442ffae72fcc9c53b907d, user_id=04e442bac7b44c819467fca141c0220b, role_id=49e47cd157d743f6905f1db07ec878a7)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:52:05.324467 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a70c9d-6392-4e9a-91ab-e863610fc87f admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:52:05.330796 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:52:05.335151 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1280/2559] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Aug 24 06:52:05 2022] PUT /identity/v3/projects/528a8395e2d442ffae72fcc9c53b907d/users/04e442bac7b44c819467fca141c0220b/roles/49e47cd157d743f6905f1db07ec878a7 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:52:05.352623 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aed00b2-0052-412a-b4d2-622e1762ea17 None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:05.352896 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aed00b2-0052-412a-b4d2-622e1762ea17 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:05.353168 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aed00b2-0052-412a-b4d2-622e1762ea17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:05.381660 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-1aed00b2-0052-412a-b4d2-622e1762ea17 None None] MFA Rules not processed for user `04e442bac7b44c819467fca141c0220b`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:52:05.382902 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-1aed00b2-0052-412a-b4d2-622e1762ea17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:52:05.400747 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:52:05.453582 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1280/2560] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:52:05 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:52:05.465007 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a70c9d-6392-4e9a-91ab-e863610fc87f admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:52:05.528270 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a70c9d-6392-4e9a-91ab-e863610fc87f admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:52:05.532272 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f141d10-207b-4a32-aba6-ed85a54b9c26 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c4def55ac0e48b29836bc5c64bc7402', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:52:05.532895 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f141d10-207b-4a32-aba6-ed85a54b9c26 service cinder] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:05.533177 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f141d10-207b-4a32-aba6-ed85a54b9c26 service cinder] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:05.533452 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f141d10-207b-4a32-aba6-ed85a54b9c26 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:05.542620 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f141d10-207b-4a32-aba6-ed85a54b9c26 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:52:05.543996 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f141d10-207b-4a32-aba6-ed85a54b9c26 service cinder] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:52:05.553882 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1281/2561] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:52:05 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:52:06.132015 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aed00b2-0052-412a-b4d2-622e1762ea17 None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:52:06.194204 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aed00b2-0052-412a-b4d2-622e1762ea17 None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:52:06.199285 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a7ff72-f1ab-4896-a22e-5f6052d0cd08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:52:06.199907 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a7ff72-f1ab-4896-a22e-5f6052d0cd08 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:06.200180 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a7ff72-f1ab-4896-a22e-5f6052d0cd08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:06.200448 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a7ff72-f1ab-4896-a22e-5f6052d0cd08 admin admin] PATH_INFO: `/v3/users/a5943c9c24024f67b78f1db5241f1ef5` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:06.201612 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a7ff72-f1ab-4896-a22e-5f6052d0cd08 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a5943c9c24024f67b78f1db5241f1ef5)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:52:06.203529 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a7ff72-f1ab-4896-a22e-5f6052d0cd08 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:52:06.389159 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-d7a7ff72-f1ab-4896-a22e-5f6052d0cd08 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a5943c9c24024f67b78f1db5241f1ef5'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:52:06.396221 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-d7a7ff72-f1ab-4896-a22e-5f6052d0cd08 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a5943c9c24024f67b78f1db5241f1ef5'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:52:06.396913 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-d7a7ff72-f1ab-4896-a22e-5f6052d0cd08 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a5943c9c24024f67b78f1db5241f1ef5'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:52:06.397583 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-d7a7ff72-f1ab-4896-a22e-5f6052d0cd08 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a5943c9c24024f67b78f1db5241f1ef5'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:52:06.494304 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-d7a7ff72-f1ab-4896-a22e-5f6052d0cd08 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a5943c9c24024f67b78f1db5241f1ef5'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:52:06.521491 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1281/2562] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Aug 24 06:52:06 2022] DELETE /identity/v3/users/a5943c9c24024f67b78f1db5241f1ef5 => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:52:06.530620 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f977f04c-5f2c-4d0d-ad2b-a1500cfdc76f None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:06.531100 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f977f04c-5f2c-4d0d-ad2b-a1500cfdc76f None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:06.531179 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f977f04c-5f2c-4d0d-ad2b-a1500cfdc76f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:06.550454 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-f977f04c-5f2c-4d0d-ad2b-a1500cfdc76f None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:52:06.551665 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-f977f04c-5f2c-4d0d-ad2b-a1500cfdc76f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:52:06.611944 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1282/2563] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:52:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:52:06.622440 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a7ff72-f1ab-4896-a22e-5f6052d0cd08 admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:52:06.624064 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-d7a7ff72-f1ab-4896-a22e-5f6052d0cd08 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:52:06.660069 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:52:06.682467 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a7ff72-f1ab-4896-a22e-5f6052d0cd08 admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:52:06.687073 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31769cf-7b26-479e-9194-86504b296b2a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bb6c79db75e4d169331c02133d1ec32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:52:06.687786 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31769cf-7b26-479e-9194-86504b296b2a service neutron] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:06.688134 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31769cf-7b26-479e-9194-86504b296b2a service neutron] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:06.688408 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31769cf-7b26-479e-9194-86504b296b2a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:06.698487 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31769cf-7b26-479e-9194-86504b296b2a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:52:06.700052 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31769cf-7b26-479e-9194-86504b296b2a service neutron] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:52:06.711617 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1282/2564] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:52:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:52:07.096047 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f977f04c-5f2c-4d0d-ad2b-a1500cfdc76f None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:52:07.097786 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-f977f04c-5f2c-4d0d-ad2b-a1500cfdc76f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:52:07.111999 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f977f04c-5f2c-4d0d-ad2b-a1500cfdc76f None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:52:07.117208 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace461cc-44f7-4a82-a019-22bbe1d3e9fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:52:07.117874 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace461cc-44f7-4a82-a019-22bbe1d3e9fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:07.118156 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace461cc-44f7-4a82-a019-22bbe1d3e9fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:07.118426 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace461cc-44f7-4a82-a019-22bbe1d3e9fd admin admin] PATH_INFO: `/v3/projects/f95c9b09fc2f47baa1d8c0b08a54e7e2` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:07.119564 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace461cc-44f7-4a82-a019-22bbe1d3e9fd admin admin] RBAC: Authorizing `identity:delete_project(project_id=f95c9b09fc2f47baa1d8c0b08a54e7e2)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:52:07.121582 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace461cc-44f7-4a82-a019-22bbe1d3e9fd admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:52:07.215434 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-ace461cc-44f7-4a82-a019-22bbe1d3e9fd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f95c9b09fc2f47baa1d8c0b08a54e7e2', 'request_id': 'req-ace461cc-44f7-4a82-a019-22bbe1d3e9fd'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:52:07.221272 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-ace461cc-44f7-4a82-a019-22bbe1d3e9fd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f95c9b09fc2f47baa1d8c0b08a54e7e2', 'request_id': 'req-ace461cc-44f7-4a82-a019-22bbe1d3e9fd'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:52:07.227614 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-ace461cc-44f7-4a82-a019-22bbe1d3e9fd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f95c9b09fc2f47baa1d8c0b08a54e7e2', 'request_id': 'req-ace461cc-44f7-4a82-a019-22bbe1d3e9fd'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:52:07.314323 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-ace461cc-44f7-4a82-a019-22bbe1d3e9fd admin admin] The token cache is being invalidate because project f95c9b09fc2f47baa1d8c0b08a54e7e2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81852) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:52:07.314702 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-ace461cc-44f7-4a82-a019-22bbe1d3e9fd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:52:07.315393 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-ace461cc-44f7-4a82-a019-22bbe1d3e9fd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:52:07.317066 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1283/2565] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Aug 24 06:52:07 2022] DELETE /identity/v3/projects/f95c9b09fc2f47baa1d8c0b08a54e7e2 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:52:07.321281 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31769cf-7b26-479e-9194-86504b296b2a service neutron] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:52:07.330935 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31769cf-7b26-479e-9194-86504b296b2a service neutron] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:52:07.388694 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c468b927-d6d9-462d-8fef-d0ab30dcbe87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:52:07.389380 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c468b927-d6d9-462d-8fef-d0ab30dcbe87 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:07.389660 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c468b927-d6d9-462d-8fef-d0ab30dcbe87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:07.389935 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c468b927-d6d9-462d-8fef-d0ab30dcbe87 admin admin] PATH_INFO: `/v3/users/04e442bac7b44c819467fca141c0220b` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:07.391076 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c468b927-d6d9-462d-8fef-d0ab30dcbe87 admin admin] RBAC: Authorizing `identity:delete_user(user_id=04e442bac7b44c819467fca141c0220b)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:52:07.393042 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c468b927-d6d9-462d-8fef-d0ab30dcbe87 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:52:07.568754 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-c468b927-d6d9-462d-8fef-d0ab30dcbe87 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '04e442bac7b44c819467fca141c0220b'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:52:07.576282 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-c468b927-d6d9-462d-8fef-d0ab30dcbe87 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '04e442bac7b44c819467fca141c0220b'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:52:07.577083 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-c468b927-d6d9-462d-8fef-d0ab30dcbe87 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '04e442bac7b44c819467fca141c0220b'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:52:07.577668 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-c468b927-d6d9-462d-8fef-d0ab30dcbe87 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '04e442bac7b44c819467fca141c0220b'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:52:07.659674 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-c468b927-d6d9-462d-8fef-d0ab30dcbe87 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '04e442bac7b44c819467fca141c0220b'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:52:07.679705 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1283/2566] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Aug 24 06:52:07 2022] DELETE /identity/v3/users/04e442bac7b44c819467fca141c0220b => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:52:08.110908 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace461cc-44f7-4a82-a019-22bbe1d3e9fd admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:52:08.112409 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-ace461cc-44f7-4a82-a019-22bbe1d3e9fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:52:08.126617 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace461cc-44f7-4a82-a019-22bbe1d3e9fd admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:52:08.153239 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:52:08.186892 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2227f437-27cb-498a-957d-14a59b678a5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:52:08.187532 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2227f437-27cb-498a-957d-14a59b678a5a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:08.187809 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2227f437-27cb-498a-957d-14a59b678a5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:08.188090 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2227f437-27cb-498a-957d-14a59b678a5a admin admin] PATH_INFO: `/v3/projects/528a8395e2d442ffae72fcc9c53b907d` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:08.189249 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2227f437-27cb-498a-957d-14a59b678a5a admin admin] RBAC: Authorizing `identity:delete_project(project_id=528a8395e2d442ffae72fcc9c53b907d)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:52:08.191193 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2227f437-27cb-498a-957d-14a59b678a5a admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:52:08.326654 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-2227f437-27cb-498a-957d-14a59b678a5a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '528a8395e2d442ffae72fcc9c53b907d', 'request_id': 'req-2227f437-27cb-498a-957d-14a59b678a5a'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:52:08.332009 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-2227f437-27cb-498a-957d-14a59b678a5a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '528a8395e2d442ffae72fcc9c53b907d', 'request_id': 'req-2227f437-27cb-498a-957d-14a59b678a5a'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:52:08.338090 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-2227f437-27cb-498a-957d-14a59b678a5a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '528a8395e2d442ffae72fcc9c53b907d', 'request_id': 'req-2227f437-27cb-498a-957d-14a59b678a5a'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:52:08.438058 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-2227f437-27cb-498a-957d-14a59b678a5a admin admin] The token cache is being invalidate because project 528a8395e2d442ffae72fcc9c53b907d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81852) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:52:08.438343 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-2227f437-27cb-498a-957d-14a59b678a5a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:52:08.438990 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-2227f437-27cb-498a-957d-14a59b678a5a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:52:08.440588 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1284/2567] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Aug 24 06:52:08 2022] DELETE /identity/v3/projects/528a8395e2d442ffae72fcc9c53b907d => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:52:08.445569 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bb9708-7007-4440-b2a2-5ae985173758 None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:08.445852 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bb9708-7007-4440-b2a2-5ae985173758 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:08.446125 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bb9708-7007-4440-b2a2-5ae985173758 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:08.465763 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-81bb9708-7007-4440-b2a2-5ae985173758 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:52:08.466955 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-81bb9708-7007-4440-b2a2-5ae985173758 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:52:08.516367 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:52:08.527425 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1284/2568] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:52:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:52:08.535239 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311021d3-26b8-4a1d-bec3-65f87540c1dd None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:08.535527 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311021d3-26b8-4a1d-bec3-65f87540c1dd None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:08.535804 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311021d3-26b8-4a1d-bec3-65f87540c1dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:08.557037 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-311021d3-26b8-4a1d-bec3-65f87540c1dd None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:52:08.558304 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-311021d3-26b8-4a1d-bec3-65f87540c1dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:52:08.566892 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1285/2569] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:52:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:52:08.571556 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bb9708-7007-4440-b2a2-5ae985173758 None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:52:08.581805 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bb9708-7007-4440-b2a2-5ae985173758 None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:52:08.587036 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cc55a5-fd05-4a00-ae08-8eb8ea77a1d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:52:08.592136 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cc55a5-fd05-4a00-ae08-8eb8ea77a1d4 admin admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:08.592136 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cc55a5-fd05-4a00-ae08-8eb8ea77a1d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:08.592136 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cc55a5-fd05-4a00-ae08-8eb8ea77a1d4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:08.592136 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31cc55a5-fd05-4a00-ae08-8eb8ea77a1d4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:52:08.592136 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31cc55a5-fd05-4a00-ae08-8eb8ea77a1d4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:52:08.592136 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31cc55a5-fd05-4a00-ae08-8eb8ea77a1d4 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:52:08.603337 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1285/2570] 10.222.0.62 () {58 vars in 1153 bytes} [Wed Aug 24 06:52:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:52:08.608540 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311021d3-26b8-4a1d-bec3-65f87540c1dd None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:52:08.618037 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311021d3-26b8-4a1d-bec3-65f87540c1dd None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:52:08.622933 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617ccb7c-c102-4ae5-ab24-49c7adaccdf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:52:08.623561 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617ccb7c-c102-4ae5-ab24-49c7adaccdf9 admin admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:08.623839 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617ccb7c-c102-4ae5-ab24-49c7adaccdf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:08.624109 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617ccb7c-c102-4ae5-ab24-49c7adaccdf9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:08.624772 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617ccb7c-c102-4ae5-ab24-49c7adaccdf9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:52:08.626656 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617ccb7c-c102-4ae5-ab24-49c7adaccdf9 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:52:08.748346 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1286/2571] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Aug 24 06:52:08 2022] POST /identity/v3/projects => generated 385 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:52:08.749558 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cc55a5-fd05-4a00-ae08-8eb8ea77a1d4 admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:52:08.759792 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cc55a5-fd05-4a00-ae08-8eb8ea77a1d4 admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:52:08.819582 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a991d9bf-224f-4c88-883f-1eab4d5e1906 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:52:08.820581 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a991d9bf-224f-4c88-883f-1eab4d5e1906 admin admin] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:08.820581 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a991d9bf-224f-4c88-883f-1eab4d5e1906 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:08.821735 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a991d9bf-224f-4c88-883f-1eab4d5e1906 admin admin] PATH_INFO: `/v3/users` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:08.821735 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a991d9bf-224f-4c88-883f-1eab4d5e1906 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:52:08.823559 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a991d9bf-224f-4c88-883f-1eab4d5e1906 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:52:08.954942 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1286/2572] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Aug 24 06:52:08 2022] POST /identity/v3/users => generated 293 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 06:52:08.959869 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617ccb7c-c102-4ae5-ab24-49c7adaccdf9 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:52:08.970926 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617ccb7c-c102-4ae5-ab24-49c7adaccdf9 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:52:08.975849 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b1bf1a-9ac8-436a-9b3f-edcaf2f72d30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:52:08.976512 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b1bf1a-9ac8-436a-9b3f-edcaf2f72d30 admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:08.976819 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b1bf1a-9ac8-436a-9b3f-edcaf2f72d30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:08.977102 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b1bf1a-9ac8-436a-9b3f-edcaf2f72d30 admin admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:08.977851 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b1bf1a-9ac8-436a-9b3f-edcaf2f72d30 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:52:08.979565 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b1bf1a-9ac8-436a-9b3f-edcaf2f72d30 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:52:08.990125 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1287/2573] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Aug 24 06:52:08 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:52:08.993961 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a991d9bf-224f-4c88-883f-1eab4d5e1906 admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:52:09.004462 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a991d9bf-224f-4c88-883f-1eab4d5e1906 admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:52:09.009322 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a16afb-b9df-4168-b46d-521e5b6eeaa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:52:09.009930 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a16afb-b9df-4168-b46d-521e5b6eeaa7 admin admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:09.010206 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a16afb-b9df-4168-b46d-521e5b6eeaa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:09.015200 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a16afb-b9df-4168-b46d-521e5b6eeaa7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:09.015200 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a16afb-b9df-4168-b46d-521e5b6eeaa7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:52:09.015200 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a16afb-b9df-4168-b46d-521e5b6eeaa7 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:52:09.024329 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1287/2574] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Aug 24 06:52:08 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:52:09.029269 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b1bf1a-9ac8-436a-9b3f-edcaf2f72d30 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:52:09.039570 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b1bf1a-9ac8-436a-9b3f-edcaf2f72d30 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:52:09.044220 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c98748-5406-434b-8c4b-97ef0465f7a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:52:09.044901 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c98748-5406-434b-8c4b-97ef0465f7a3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:09.045181 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c98748-5406-434b-8c4b-97ef0465f7a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:09.045476 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c98748-5406-434b-8c4b-97ef0465f7a3 admin admin] PATH_INFO: `/v3/projects/e913c90289df4d32b4ad4843943e78ea/users/6f7ca7111298490da3d664ea309194ac/roles/580aa7dbb38d4cff8f9d1bb8983f99ef` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:09.061154 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c98748-5406-434b-8c4b-97ef0465f7a3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e913c90289df4d32b4ad4843943e78ea, user_id=6f7ca7111298490da3d664ea309194ac, role_id=580aa7dbb38d4cff8f9d1bb8983f99ef)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:52:09.064367 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c98748-5406-434b-8c4b-97ef0465f7a3 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:52:09.149852 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1288/2575] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Aug 24 06:52:09 2022] PUT /identity/v3/projects/e913c90289df4d32b4ad4843943e78ea/users/6f7ca7111298490da3d664ea309194ac/roles/580aa7dbb38d4cff8f9d1bb8983f99ef => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:52:09.164216 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62049060-59bf-40cc-9272-8648124fbb3d None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:09.164519 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62049060-59bf-40cc-9272-8648124fbb3d None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:09.164868 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62049060-59bf-40cc-9272-8648124fbb3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:09.193130 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-62049060-59bf-40cc-9272-8648124fbb3d None None] MFA Rules not processed for user `6f7ca7111298490da3d664ea309194ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:52:09.194348 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-62049060-59bf-40cc-9272-8648124fbb3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:52:09.258860 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1288/2576] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:52:09 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:52:09.270383 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c98748-5406-434b-8c4b-97ef0465f7a3 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:52:09.272063 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-22c98748-5406-434b-8c4b-97ef0465f7a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:52:09.328771 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c98748-5406-434b-8c4b-97ef0465f7a3 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:52:09.332374 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927be58f-8e4a-4f9c-94a8-d488dea18cd7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c4def55ac0e48b29836bc5c64bc7402', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:52:09.332995 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927be58f-8e4a-4f9c-94a8-d488dea18cd7 service cinder] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:09.333268 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927be58f-8e4a-4f9c-94a8-d488dea18cd7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:09.333533 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927be58f-8e4a-4f9c-94a8-d488dea18cd7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:09.339685 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:52:09.341955 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927be58f-8e4a-4f9c-94a8-d488dea18cd7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:52:09.343374 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927be58f-8e4a-4f9c-94a8-d488dea18cd7 service cinder] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:52:09.352081 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1289/2577] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:52:09 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:52:16.262012 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62049060-59bf-40cc-9272-8648124fbb3d None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:52:16.263348 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-62049060-59bf-40cc-9272-8648124fbb3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:52:16.318352 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:52:16.341188 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62049060-59bf-40cc-9272-8648124fbb3d None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:52:16.346592 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4144247-6ac8-497c-a88d-b31b3fb20638 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:52:16.347422 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4144247-6ac8-497c-a88d-b31b3fb20638 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:16.347513 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4144247-6ac8-497c-a88d-b31b3fb20638 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:16.347775 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4144247-6ac8-497c-a88d-b31b3fb20638 admin admin] PATH_INFO: `/v3/users/b9528268491641fcb78f37c2bb35e328` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:16.348959 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4144247-6ac8-497c-a88d-b31b3fb20638 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b9528268491641fcb78f37c2bb35e328)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:52:16.350875 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4144247-6ac8-497c-a88d-b31b3fb20638 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:52:16.612278 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-f4144247-6ac8-497c-a88d-b31b3fb20638 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b9528268491641fcb78f37c2bb35e328'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:52:16.619217 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-f4144247-6ac8-497c-a88d-b31b3fb20638 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b9528268491641fcb78f37c2bb35e328'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:52:16.619874 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-f4144247-6ac8-497c-a88d-b31b3fb20638 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b9528268491641fcb78f37c2bb35e328'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:52:16.620487 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-f4144247-6ac8-497c-a88d-b31b3fb20638 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b9528268491641fcb78f37c2bb35e328'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:52:16.745107 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-f4144247-6ac8-497c-a88d-b31b3fb20638 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b9528268491641fcb78f37c2bb35e328'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:52:16.765854 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1289/2578] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Aug 24 06:52:16 2022] DELETE /identity/v3/users/b9528268491641fcb78f37c2bb35e328 => generated 0 bytes in 505 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:52:16.780755 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587fa4a7-c9d8-4e0c-b1a5-c1fcff4590c1 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:16.780755 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587fa4a7-c9d8-4e0c-b1a5-c1fcff4590c1 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:16.780755 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587fa4a7-c9d8-4e0c-b1a5-c1fcff4590c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:16.791690 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-587fa4a7-c9d8-4e0c-b1a5-c1fcff4590c1 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:52:16.792950 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-587fa4a7-c9d8-4e0c-b1a5-c1fcff4590c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:52:16.865134 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1290/2579] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:52:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:52:16.875888 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4144247-6ac8-497c-a88d-b31b3fb20638 admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:52:16.877413 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-f4144247-6ac8-497c-a88d-b31b3fb20638 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:52:16.936869 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4144247-6ac8-497c-a88d-b31b3fb20638 admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:52:16.940522 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa31c7e-1ece-4258-8fcd-69e7dcf54277 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bb6c79db75e4d169331c02133d1ec32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:52:16.941149 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa31c7e-1ece-4258-8fcd-69e7dcf54277 service neutron] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:16.941425 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa31c7e-1ece-4258-8fcd-69e7dcf54277 service neutron] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:16.941693 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa31c7e-1ece-4258-8fcd-69e7dcf54277 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:16.952401 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa31c7e-1ece-4258-8fcd-69e7dcf54277 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:52:16.953899 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa31c7e-1ece-4258-8fcd-69e7dcf54277 service neutron] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:52:16.964081 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1290/2580] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:52:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:52:17.394256 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587fa4a7-c9d8-4e0c-b1a5-c1fcff4590c1 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:52:17.398632 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-587fa4a7-c9d8-4e0c-b1a5-c1fcff4590c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:52:17.444557 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587fa4a7-c9d8-4e0c-b1a5-c1fcff4590c1 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:52:17.451304 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f754766-db2e-4df6-b30d-43c74225b18c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:52:17.451945 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f754766-db2e-4df6-b30d-43c74225b18c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:17.452220 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f754766-db2e-4df6-b30d-43c74225b18c admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:17.452508 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f754766-db2e-4df6-b30d-43c74225b18c admin admin] PATH_INFO: `/v3/projects/6118a695540f486cbad61fe0d566e457` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:17.453634 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f754766-db2e-4df6-b30d-43c74225b18c admin admin] RBAC: Authorizing `identity:delete_project(project_id=6118a695540f486cbad61fe0d566e457)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:52:17.455584 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f754766-db2e-4df6-b30d-43c74225b18c admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:52:17.576188 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-0f754766-db2e-4df6-b30d-43c74225b18c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6118a695540f486cbad61fe0d566e457', 'request_id': 'req-0f754766-db2e-4df6-b30d-43c74225b18c'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:52:17.582555 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-0f754766-db2e-4df6-b30d-43c74225b18c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6118a695540f486cbad61fe0d566e457', 'request_id': 'req-0f754766-db2e-4df6-b30d-43c74225b18c'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:52:17.589093 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-0f754766-db2e-4df6-b30d-43c74225b18c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6118a695540f486cbad61fe0d566e457', 'request_id': 'req-0f754766-db2e-4df6-b30d-43c74225b18c'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:52:17.676609 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-0f754766-db2e-4df6-b30d-43c74225b18c admin admin] The token cache is being invalidate because project 6118a695540f486cbad61fe0d566e457 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81852) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:52:17.676904 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-0f754766-db2e-4df6-b30d-43c74225b18c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:52:17.677671 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-0f754766-db2e-4df6-b30d-43c74225b18c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:52:17.679336 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1291/2581] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Aug 24 06:52:17 2022] DELETE /identity/v3/projects/6118a695540f486cbad61fe0d566e457 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 06:52:18.259375 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa31c7e-1ece-4258-8fcd-69e7dcf54277 service neutron] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:52:18.260871 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-eaa31c7e-1ece-4258-8fcd-69e7dcf54277 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:52:18.274153 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa31c7e-1ece-4258-8fcd-69e7dcf54277 service neutron] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:52:18.344981 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2fa36e-6ab1-481e-8a78-d1281fa8d71a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:52:18.345757 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2fa36e-6ab1-481e-8a78-d1281fa8d71a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:18.346111 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2fa36e-6ab1-481e-8a78-d1281fa8d71a admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:18.346387 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2fa36e-6ab1-481e-8a78-d1281fa8d71a admin admin] PATH_INFO: `/v3/users/bcd35ac12b4145419178e7d538ba8b0e` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:18.347510 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2fa36e-6ab1-481e-8a78-d1281fa8d71a admin admin] RBAC: Authorizing `identity:delete_user(user_id=bcd35ac12b4145419178e7d538ba8b0e)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:52:18.349469 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2fa36e-6ab1-481e-8a78-d1281fa8d71a admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:52:18.363816 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:52:18.599170 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-ab2fa36e-6ab1-481e-8a78-d1281fa8d71a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bcd35ac12b4145419178e7d538ba8b0e'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:52:18.607061 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-ab2fa36e-6ab1-481e-8a78-d1281fa8d71a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bcd35ac12b4145419178e7d538ba8b0e'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:52:18.607747 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-ab2fa36e-6ab1-481e-8a78-d1281fa8d71a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bcd35ac12b4145419178e7d538ba8b0e'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:52:18.608354 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-ab2fa36e-6ab1-481e-8a78-d1281fa8d71a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bcd35ac12b4145419178e7d538ba8b0e'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:52:18.714729 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-ab2fa36e-6ab1-481e-8a78-d1281fa8d71a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bcd35ac12b4145419178e7d538ba8b0e'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:52:18.732815 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1291/2582] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Aug 24 06:52:18 2022] DELETE /identity/v3/users/bcd35ac12b4145419178e7d538ba8b0e => generated 0 bytes in 474 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:52:18.737552 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffaf7ca8-c02d-423b-aa1a-49e00af07c66 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:18.738051 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffaf7ca8-c02d-423b-aa1a-49e00af07c66 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:18.738135 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffaf7ca8-c02d-423b-aa1a-49e00af07c66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:18.757309 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-ffaf7ca8-c02d-423b-aa1a-49e00af07c66 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:52:18.758520 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-ffaf7ca8-c02d-423b-aa1a-49e00af07c66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:52:18.817576 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1292/2583] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:52:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 06:52:18.828176 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2fa36e-6ab1-481e-8a78-d1281fa8d71a admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:52:18.829673 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-ab2fa36e-6ab1-481e-8a78-d1281fa8d71a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:52:18.887679 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2fa36e-6ab1-481e-8a78-d1281fa8d71a admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:52:18.891192 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cb6467-653f-4808-bf57-78dd08779eed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bb6c79db75e4d169331c02133d1ec32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:52:18.891797 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cb6467-653f-4808-bf57-78dd08779eed service neutron] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:18.892086 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cb6467-653f-4808-bf57-78dd08779eed service neutron] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:18.892355 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cb6467-653f-4808-bf57-78dd08779eed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:18.902357 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07cb6467-653f-4808-bf57-78dd08779eed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:52:18.903791 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07cb6467-653f-4808-bf57-78dd08779eed service neutron] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:52:18.914240 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1292/2584] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:52:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:52:19.389165 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffaf7ca8-c02d-423b-aa1a-49e00af07c66 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:52:19.390796 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-ffaf7ca8-c02d-423b-aa1a-49e00af07c66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:52:19.413293 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffaf7ca8-c02d-423b-aa1a-49e00af07c66 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:52:19.418686 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1632bf66-dc21-4ba3-bf34-f46d9caef461 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:52:19.419347 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1632bf66-dc21-4ba3-bf34-f46d9caef461 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:19.419624 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1632bf66-dc21-4ba3-bf34-f46d9caef461 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:19.419900 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1632bf66-dc21-4ba3-bf34-f46d9caef461 admin admin] PATH_INFO: `/v3/projects/4c625f068e0e4f2e8df02672171adb72` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:19.421048 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1632bf66-dc21-4ba3-bf34-f46d9caef461 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4c625f068e0e4f2e8df02672171adb72)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:52:19.422953 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1632bf66-dc21-4ba3-bf34-f46d9caef461 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:52:19.441088 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:52:19.548321 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-1632bf66-dc21-4ba3-bf34-f46d9caef461 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4c625f068e0e4f2e8df02672171adb72', 'request_id': 'req-1632bf66-dc21-4ba3-bf34-f46d9caef461'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:52:19.553777 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-1632bf66-dc21-4ba3-bf34-f46d9caef461 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4c625f068e0e4f2e8df02672171adb72', 'request_id': 'req-1632bf66-dc21-4ba3-bf34-f46d9caef461'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:52:19.562717 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-1632bf66-dc21-4ba3-bf34-f46d9caef461 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4c625f068e0e4f2e8df02672171adb72', 'request_id': 'req-1632bf66-dc21-4ba3-bf34-f46d9caef461'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:52:19.671110 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-1632bf66-dc21-4ba3-bf34-f46d9caef461 admin admin] The token cache is being invalidate because project 4c625f068e0e4f2e8df02672171adb72 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81852) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:52:19.671456 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-1632bf66-dc21-4ba3-bf34-f46d9caef461 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:52:19.672113 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-1632bf66-dc21-4ba3-bf34-f46d9caef461 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:52:19.673678 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1293/2585] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Aug 24 06:52:19 2022] DELETE /identity/v3/projects/4c625f068e0e4f2e8df02672171adb72 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:52:19.678697 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d80a7a7-e4e4-4c7e-a9a1-3b8f2bb75aeb None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:19.679182 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d80a7a7-e4e4-4c7e-a9a1-3b8f2bb75aeb None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:19.679266 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d80a7a7-e4e4-4c7e-a9a1-3b8f2bb75aeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:19.700708 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-6d80a7a7-e4e4-4c7e-a9a1-3b8f2bb75aeb None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:52:19.701982 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-6d80a7a7-e4e4-4c7e-a9a1-3b8f2bb75aeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:52:19.767310 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1293/2586] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:52:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:52:19.775126 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd49e8f-396e-4b4c-89df-78f28033d107 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:19.775705 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd49e8f-396e-4b4c-89df-78f28033d107 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:19.775705 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd49e8f-396e-4b4c-89df-78f28033d107 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:19.798465 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-ccd49e8f-396e-4b4c-89df-78f28033d107 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:52:19.799849 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-ccd49e8f-396e-4b4c-89df-78f28033d107 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:52:19.809807 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1294/2587] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:52:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:52:19.814704 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d80a7a7-e4e4-4c7e-a9a1-3b8f2bb75aeb None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:52:19.826032 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d80a7a7-e4e4-4c7e-a9a1-3b8f2bb75aeb None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:52:19.831036 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1f5878-3ed5-40d0-bfc7-dd6821fa5ced admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:52:19.831649 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1f5878-3ed5-40d0-bfc7-dd6821fa5ced admin admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:19.831948 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1f5878-3ed5-40d0-bfc7-dd6821fa5ced admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:19.832219 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1f5878-3ed5-40d0-bfc7-dd6821fa5ced admin admin] PATH_INFO: `/v3/domains` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:19.832901 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1f5878-3ed5-40d0-bfc7-dd6821fa5ced admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:52:19.833207 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1f5878-3ed5-40d0-bfc7-dd6821fa5ced admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:52:19.834968 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1f5878-3ed5-40d0-bfc7-dd6821fa5ced admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:52:19.849284 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1294/2588] 10.222.0.62 () {58 vars in 1153 bytes} [Wed Aug 24 06:52:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:52:19.853891 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd49e8f-396e-4b4c-89df-78f28033d107 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:52:19.864313 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd49e8f-396e-4b4c-89df-78f28033d107 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:52:19.869542 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c08ceee-cc37-42a3-ac80-6e5a538a9a89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:52:19.870784 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c08ceee-cc37-42a3-ac80-6e5a538a9a89 admin admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:19.871288 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c08ceee-cc37-42a3-ac80-6e5a538a9a89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:19.871624 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c08ceee-cc37-42a3-ac80-6e5a538a9a89 admin admin] PATH_INFO: `/v3/projects` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:19.872412 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c08ceee-cc37-42a3-ac80-6e5a538a9a89 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:52:19.875117 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c08ceee-cc37-42a3-ac80-6e5a538a9a89 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:52:19.964762 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1295/2589] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Aug 24 06:52:19 2022] POST /identity/v3/projects => generated 375 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:52:19.968417 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1f5878-3ed5-40d0-bfc7-dd6821fa5ced admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:52:19.978501 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1f5878-3ed5-40d0-bfc7-dd6821fa5ced admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:52:19.997780 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:52:20.045150 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6e9ce4-e9d7-4523-a142-3d99bacc08a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:52:20.045779 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6e9ce4-e9d7-4523-a142-3d99bacc08a0 admin admin] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:20.046099 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6e9ce4-e9d7-4523-a142-3d99bacc08a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:20.046371 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6e9ce4-e9d7-4523-a142-3d99bacc08a0 admin admin] PATH_INFO: `/v3/users` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:20.047020 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6e9ce4-e9d7-4523-a142-3d99bacc08a0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:52:20.048992 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6e9ce4-e9d7-4523-a142-3d99bacc08a0 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:52:20.128411 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1295/2590] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Aug 24 06:52:19 2022] POST /identity/v3/users => generated 288 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:52:20.132831 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c08ceee-cc37-42a3-ac80-6e5a538a9a89 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:52:20.143862 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c08ceee-cc37-42a3-ac80-6e5a538a9a89 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:52:20.148431 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708fbe17-7b32-4ac1-8b50-03ca58bdd3e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:52:20.149058 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708fbe17-7b32-4ac1-8b50-03ca58bdd3e7 admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:20.149333 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708fbe17-7b32-4ac1-8b50-03ca58bdd3e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:20.153602 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708fbe17-7b32-4ac1-8b50-03ca58bdd3e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:20.153602 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708fbe17-7b32-4ac1-8b50-03ca58bdd3e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:52:20.153602 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708fbe17-7b32-4ac1-8b50-03ca58bdd3e7 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:52:20.161750 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1296/2591] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Aug 24 06:52:20 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:52:20.166045 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6e9ce4-e9d7-4523-a142-3d99bacc08a0 admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:52:20.177100 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6e9ce4-e9d7-4523-a142-3d99bacc08a0 admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:52:20.183070 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7039c7ba-d148-4535-b477-770632b5ce8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:52:20.183722 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7039c7ba-d148-4535-b477-770632b5ce8b admin admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:20.184005 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7039c7ba-d148-4535-b477-770632b5ce8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:20.184277 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7039c7ba-d148-4535-b477-770632b5ce8b admin admin] PATH_INFO: `/v3/roles` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:20.185214 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7039c7ba-d148-4535-b477-770632b5ce8b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:52:20.186832 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7039c7ba-d148-4535-b477-770632b5ce8b admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:52:20.197042 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1296/2592] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Aug 24 06:52:20 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:52:20.201882 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708fbe17-7b32-4ac1-8b50-03ca58bdd3e7 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:52:20.212116 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708fbe17-7b32-4ac1-8b50-03ca58bdd3e7 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:52:20.216787 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e578d236-89d4-43cb-9b8e-1fe6f12f0e29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:52:20.217431 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e578d236-89d4-43cb-9b8e-1fe6f12f0e29 admin admin] REQUEST_METHOD: `PUT` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:20.217705 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e578d236-89d4-43cb-9b8e-1fe6f12f0e29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:20.217972 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e578d236-89d4-43cb-9b8e-1fe6f12f0e29 admin admin] PATH_INFO: `/v3/projects/a7e3687c114142e69f1812dbf2613c63/users/85450e77c8334a08a7d09a44df532ce5/roles/580aa7dbb38d4cff8f9d1bb8983f99ef` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:20.232908 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e578d236-89d4-43cb-9b8e-1fe6f12f0e29 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a7e3687c114142e69f1812dbf2613c63, user_id=85450e77c8334a08a7d09a44df532ce5, role_id=580aa7dbb38d4cff8f9d1bb8983f99ef)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:52:20.235539 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e578d236-89d4-43cb-9b8e-1fe6f12f0e29 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:52:20.294606 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1297/2593] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Aug 24 06:52:20 2022] PUT /identity/v3/projects/a7e3687c114142e69f1812dbf2613c63/users/85450e77c8334a08a7d09a44df532ce5/roles/580aa7dbb38d4cff8f9d1bb8983f99ef => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 06:52:20.312306 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f13d836-5209-4be4-8ac8-4355631cdb1e None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:20.312621 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f13d836-5209-4be4-8ac8-4355631cdb1e None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:20.312896 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f13d836-5209-4be4-8ac8-4355631cdb1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:20.341451 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-0f13d836-5209-4be4-8ac8-4355631cdb1e None None] MFA Rules not processed for user `85450e77c8334a08a7d09a44df532ce5`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:52:20.342668 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-0f13d836-5209-4be4-8ac8-4355631cdb1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:52:20.406989 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1297/2594] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:52:20 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:52:20.431473 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e578d236-89d4-43cb-9b8e-1fe6f12f0e29 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:52:20.433610 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-e578d236-89d4-43cb-9b8e-1fe6f12f0e29 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:52:20.437867 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f13d836-5209-4be4-8ac8-4355631cdb1e None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:52:20.439269 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-0f13d836-5209-4be4-8ac8-4355631cdb1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:52:20.505603 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e578d236-89d4-43cb-9b8e-1fe6f12f0e29 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:52:20.510266 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3aa80a-bbfa-42b1-9cbc-c9890914aea5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:52:20.510945 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3aa80a-bbfa-42b1-9cbc-c9890914aea5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:20.511220 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3aa80a-bbfa-42b1-9cbc-c9890914aea5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:20.511491 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3aa80a-bbfa-42b1-9cbc-c9890914aea5 admin admin] PATH_INFO: `/v3/users/6f7ca7111298490da3d664ea309194ac` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:20.511585 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f13d836-5209-4be4-8ac8-4355631cdb1e None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:52:20.512672 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3aa80a-bbfa-42b1-9cbc-c9890914aea5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6f7ca7111298490da3d664ea309194ac)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:52:20.514720 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3aa80a-bbfa-42b1-9cbc-c9890914aea5 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:52:20.515264 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d329b034-da57-47ca-b5a9-656f81ef30f9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c4def55ac0e48b29836bc5c64bc7402', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:52:20.515871 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d329b034-da57-47ca-b5a9-656f81ef30f9 service cinder] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:20.516249 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d329b034-da57-47ca-b5a9-656f81ef30f9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:20.516417 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d329b034-da57-47ca-b5a9-656f81ef30f9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:20.527077 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d329b034-da57-47ca-b5a9-656f81ef30f9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:52:20.528547 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d329b034-da57-47ca-b5a9-656f81ef30f9 service cinder] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:52:20.538849 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1298/2595] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:52:20 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 06:52:20.740630 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=84,keystone:DELETE=13,keystone:INSERT=3 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:52:20.745578 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-cc3aa80a-bbfa-42b1-9cbc-c9890914aea5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6f7ca7111298490da3d664ea309194ac'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:52:20.752461 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-cc3aa80a-bbfa-42b1-9cbc-c9890914aea5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6f7ca7111298490da3d664ea309194ac'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:52:20.753164 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-cc3aa80a-bbfa-42b1-9cbc-c9890914aea5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6f7ca7111298490da3d664ea309194ac'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:52:20.861693 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-cc3aa80a-bbfa-42b1-9cbc-c9890914aea5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6f7ca7111298490da3d664ea309194ac'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:52:20.875669 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-cc3aa80a-bbfa-42b1-9cbc-c9890914aea5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6f7ca7111298490da3d664ea309194ac'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:52:20.882172 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1298/2596] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Aug 24 06:52:20 2022] DELETE /identity/v3/users/6f7ca7111298490da3d664ea309194ac => generated 0 bytes in 451 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 06:52:20.886551 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cca0867-251c-4e01-bdc1-04b4483293df None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:20.887044 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cca0867-251c-4e01-bdc1-04b4483293df None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:20.887161 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cca0867-251c-4e01-bdc1-04b4483293df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:20.907120 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-1cca0867-251c-4e01-bdc1-04b4483293df None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:52:20.908351 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-1cca0867-251c-4e01-bdc1-04b4483293df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:52:20.938953 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:52:20.972385 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1299/2597] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:52:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:52:20.982153 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3aa80a-bbfa-42b1-9cbc-c9890914aea5 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:52:20.983554 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-cc3aa80a-bbfa-42b1-9cbc-c9890914aea5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:52:21.043092 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3aa80a-bbfa-42b1-9cbc-c9890914aea5 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:52:21.046740 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f59036-86ff-4d7f-a697-062e0c2be545 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bb6c79db75e4d169331c02133d1ec32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:52:21.047340 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f59036-86ff-4d7f-a697-062e0c2be545 service neutron] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:21.047614 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f59036-86ff-4d7f-a697-062e0c2be545 service neutron] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:21.047880 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f59036-86ff-4d7f-a697-062e0c2be545 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:21.057035 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f59036-86ff-4d7f-a697-062e0c2be545 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:52:21.058472 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f59036-86ff-4d7f-a697-062e0c2be545 service neutron] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:52:21.068999 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1299/2598] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:52:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 06:52:21.489417 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cca0867-251c-4e01-bdc1-04b4483293df None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:52:21.491013 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-1cca0867-251c-4e01-bdc1-04b4483293df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:52:21.507540 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cca0867-251c-4e01-bdc1-04b4483293df None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:52:21.521035 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5d50e3-8c42-4108-8e9a-a42ba6bd0378 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:52:21.521751 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5d50e3-8c42-4108-8e9a-a42ba6bd0378 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:21.522193 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5d50e3-8c42-4108-8e9a-a42ba6bd0378 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:21.522563 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5d50e3-8c42-4108-8e9a-a42ba6bd0378 admin admin] PATH_INFO: `/v3/projects/e913c90289df4d32b4ad4843943e78ea` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:21.523853 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5d50e3-8c42-4108-8e9a-a42ba6bd0378 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e913c90289df4d32b4ad4843943e78ea)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:52:21.525950 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5d50e3-8c42-4108-8e9a-a42ba6bd0378 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:52:21.621806 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-2f5d50e3-8c42-4108-8e9a-a42ba6bd0378 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e913c90289df4d32b4ad4843943e78ea', 'request_id': 'req-2f5d50e3-8c42-4108-8e9a-a42ba6bd0378'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:52:21.627595 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-2f5d50e3-8c42-4108-8e9a-a42ba6bd0378 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e913c90289df4d32b4ad4843943e78ea', 'request_id': 'req-2f5d50e3-8c42-4108-8e9a-a42ba6bd0378'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:52:21.633988 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-2f5d50e3-8c42-4108-8e9a-a42ba6bd0378 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e913c90289df4d32b4ad4843943e78ea', 'request_id': 'req-2f5d50e3-8c42-4108-8e9a-a42ba6bd0378'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:52:21.764991 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-2f5d50e3-8c42-4108-8e9a-a42ba6bd0378 admin admin] The token cache is being invalidate because project e913c90289df4d32b4ad4843943e78ea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81851) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:52:21.765205 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-2f5d50e3-8c42-4108-8e9a-a42ba6bd0378 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:52:21.765871 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-2f5d50e3-8c42-4108-8e9a-a42ba6bd0378 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:52:21.767435 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1300/2599] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Aug 24 06:52:21 2022] DELETE /identity/v3/projects/e913c90289df4d32b4ad4843943e78ea => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:52:21.772662 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfd5315-f2f4-4b1e-b5ea-f1b98c7d1cc7 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:21.772971 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfd5315-f2f4-4b1e-b5ea-f1b98c7d1cc7 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:21.773280 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfd5315-f2f4-4b1e-b5ea-f1b98c7d1cc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:21.793081 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-3cfd5315-f2f4-4b1e-b5ea-f1b98c7d1cc7 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:52:21.794283 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-3cfd5315-f2f4-4b1e-b5ea-f1b98c7d1cc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:52:21.855231 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1300/2600] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:52:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:52:21.863895 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ba41fd-b625-42e6-9680-2f3074ffd310 None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:21.864205 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ba41fd-b625-42e6-9680-2f3074ffd310 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:21.864482 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ba41fd-b625-42e6-9680-2f3074ffd310 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:21.884600 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-01ba41fd-b625-42e6-9680-2f3074ffd310 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:52:21.885810 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-01ba41fd-b625-42e6-9680-2f3074ffd310 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:52:21.894348 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1301/2601] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:52:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:52:21.898684 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfd5315-f2f4-4b1e-b5ea-f1b98c7d1cc7 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:52:21.908413 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfd5315-f2f4-4b1e-b5ea-f1b98c7d1cc7 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:52:21.913097 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65992215-f754-4632-96cf-dc48a33c449d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:52:21.913698 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65992215-f754-4632-96cf-dc48a33c449d admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:21.913973 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65992215-f754-4632-96cf-dc48a33c449d admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:21.914242 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65992215-f754-4632-96cf-dc48a33c449d admin admin] PATH_INFO: `/v3/domains` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:21.914908 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65992215-f754-4632-96cf-dc48a33c449d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:52:21.915212 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65992215-f754-4632-96cf-dc48a33c449d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:52:21.917349 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65992215-f754-4632-96cf-dc48a33c449d admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:52:21.929186 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1301/2602] 10.222.0.62 () {58 vars in 1153 bytes} [Wed Aug 24 06:52:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:52:21.934204 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ba41fd-b625-42e6-9680-2f3074ffd310 None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:52:21.944156 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ba41fd-b625-42e6-9680-2f3074ffd310 None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:52:21.948833 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4039eaf6-8802-478a-825f-6fdb70d8bec8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:52:21.949431 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4039eaf6-8802-478a-825f-6fdb70d8bec8 admin admin] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:21.949705 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4039eaf6-8802-478a-825f-6fdb70d8bec8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:21.949969 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4039eaf6-8802-478a-825f-6fdb70d8bec8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:21.950579 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4039eaf6-8802-478a-825f-6fdb70d8bec8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:52:21.952472 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4039eaf6-8802-478a-825f-6fdb70d8bec8 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:52:22.289979 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1302/2603] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Aug 24 06:52:21 2022] POST /identity/v3/projects => generated 375 bytes in 353 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:52:22.291048 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65992215-f754-4632-96cf-dc48a33c449d admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:52:22.301787 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65992215-f754-4632-96cf-dc48a33c449d admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:52:22.349367 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:52:22.361705 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2478564-8db9-425e-8ba9-43246b0ae64a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:52:22.362295 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2478564-8db9-425e-8ba9-43246b0ae64a admin admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:22.362567 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2478564-8db9-425e-8ba9-43246b0ae64a admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:22.362838 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2478564-8db9-425e-8ba9-43246b0ae64a admin admin] PATH_INFO: `/v3/users` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:22.363461 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2478564-8db9-425e-8ba9-43246b0ae64a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:52:22.365410 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2478564-8db9-425e-8ba9-43246b0ae64a admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:52:22.466301 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1302/2604] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Aug 24 06:52:22 2022] POST /identity/v3/users => generated 288 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:52:22.470455 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4039eaf6-8802-478a-825f-6fdb70d8bec8 admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:52:22.481097 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4039eaf6-8802-478a-825f-6fdb70d8bec8 admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:52:22.485715 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec724c2e-a2c9-4df7-9b64-9d8190f14504 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:52:22.486256 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec724c2e-a2c9-4df7-9b64-9d8190f14504 admin admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:22.486536 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec724c2e-a2c9-4df7-9b64-9d8190f14504 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:22.486809 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec724c2e-a2c9-4df7-9b64-9d8190f14504 admin admin] PATH_INFO: `/v3/roles` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:22.487451 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec724c2e-a2c9-4df7-9b64-9d8190f14504 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:52:22.489163 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec724c2e-a2c9-4df7-9b64-9d8190f14504 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:52:22.499455 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1303/2605] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Aug 24 06:52:22 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:52:22.504644 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2478564-8db9-425e-8ba9-43246b0ae64a admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:52:22.515337 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2478564-8db9-425e-8ba9-43246b0ae64a admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:52:22.520245 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447662aa-247b-4110-8e1c-85d40259d47e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:52:22.520790 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447662aa-247b-4110-8e1c-85d40259d47e admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:22.521068 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447662aa-247b-4110-8e1c-85d40259d47e admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:22.521343 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447662aa-247b-4110-8e1c-85d40259d47e admin admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:22.521984 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447662aa-247b-4110-8e1c-85d40259d47e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:52:22.523891 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447662aa-247b-4110-8e1c-85d40259d47e admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:52:22.540186 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1303/2606] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Aug 24 06:52:22 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:52:22.541694 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec724c2e-a2c9-4df7-9b64-9d8190f14504 admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:52:22.551713 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec724c2e-a2c9-4df7-9b64-9d8190f14504 admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:52:22.556423 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11caaa57-329e-4cdb-af7b-86bde1ba6748 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:52:22.559463 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11caaa57-329e-4cdb-af7b-86bde1ba6748 admin admin] REQUEST_METHOD: `PUT` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:22.559463 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11caaa57-329e-4cdb-af7b-86bde1ba6748 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:22.559463 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11caaa57-329e-4cdb-af7b-86bde1ba6748 admin admin] PATH_INFO: `/v3/projects/2a3a7c908b22404ba3159609cb5c7c69/users/ea128ec658964a1ba7a3ada90359c847/roles/580aa7dbb38d4cff8f9d1bb8983f99ef` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:22.573969 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11caaa57-329e-4cdb-af7b-86bde1ba6748 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a3a7c908b22404ba3159609cb5c7c69, user_id=ea128ec658964a1ba7a3ada90359c847, role_id=580aa7dbb38d4cff8f9d1bb8983f99ef)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:52:22.576728 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11caaa57-329e-4cdb-af7b-86bde1ba6748 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:52:22.694076 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1304/2607] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Aug 24 06:52:22 2022] PUT /identity/v3/projects/2a3a7c908b22404ba3159609cb5c7c69/users/ea128ec658964a1ba7a3ada90359c847/roles/580aa7dbb38d4cff8f9d1bb8983f99ef => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:52:22.705225 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bb4f2d-7d40-4b33-9f84-fb463f9e33cd None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:22.705505 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bb4f2d-7d40-4b33-9f84-fb463f9e33cd None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:22.705777 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bb4f2d-7d40-4b33-9f84-fb463f9e33cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:22.735196 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-12bb4f2d-7d40-4b33-9f84-fb463f9e33cd None None] MFA Rules not processed for user `ea128ec658964a1ba7a3ada90359c847`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:52:22.737664 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-12bb4f2d-7d40-4b33-9f84-fb463f9e33cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:52:22.813386 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1304/2608] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:52:22 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:52:22.817019 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11caaa57-329e-4cdb-af7b-86bde1ba6748 admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:52:22.870771 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:52:22.881256 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11caaa57-329e-4cdb-af7b-86bde1ba6748 admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:52:22.887514 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd2bca5-3506-49fc-bd34-6b1fd28657f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:52:22.888134 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd2bca5-3506-49fc-bd34-6b1fd28657f7 admin admin] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:22.888419 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd2bca5-3506-49fc-bd34-6b1fd28657f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:22.888753 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd2bca5-3506-49fc-bd34-6b1fd28657f7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:22.889488 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd2bca5-3506-49fc-bd34-6b1fd28657f7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:52:22.891420 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd2bca5-3506-49fc-bd34-6b1fd28657f7 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:52:22.996326 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1305/2609] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Aug 24 06:52:22 2022] POST /identity/v3/projects => generated 375 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:52:23.000275 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bb4f2d-7d40-4b33-9f84-fb463f9e33cd None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:52:23.011709 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bb4f2d-7d40-4b33-9f84-fb463f9e33cd None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:52:23.068997 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90068251-5766-4ac5-add3-d4ba99da58d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:52:23.069607 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90068251-5766-4ac5-add3-d4ba99da58d4 admin admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:23.069883 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90068251-5766-4ac5-add3-d4ba99da58d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:23.070150 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90068251-5766-4ac5-add3-d4ba99da58d4 admin admin] PATH_INFO: `/v3/users` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:23.070809 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90068251-5766-4ac5-add3-d4ba99da58d4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:52:23.073460 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90068251-5766-4ac5-add3-d4ba99da58d4 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:52:23.171937 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1305/2610] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Aug 24 06:52:22 2022] POST /identity/v3/users => generated 294 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:52:23.175949 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd2bca5-3506-49fc-bd34-6b1fd28657f7 admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:52:23.187330 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd2bca5-3506-49fc-bd34-6b1fd28657f7 admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:52:23.192616 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266fd535-d7ac-4fa9-b15f-7710ea98d726 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:52:23.195516 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266fd535-d7ac-4fa9-b15f-7710ea98d726 admin admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:23.195516 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266fd535-d7ac-4fa9-b15f-7710ea98d726 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:23.195516 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266fd535-d7ac-4fa9-b15f-7710ea98d726 admin admin] PATH_INFO: `/v3/roles` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:23.195516 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266fd535-d7ac-4fa9-b15f-7710ea98d726 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:52:23.196249 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266fd535-d7ac-4fa9-b15f-7710ea98d726 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:52:23.210320 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1306/2611] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Aug 24 06:52:23 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:52:23.214631 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90068251-5766-4ac5-add3-d4ba99da58d4 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:52:23.224632 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90068251-5766-4ac5-add3-d4ba99da58d4 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:52:23.230594 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9a062d-e186-4347-9efe-e06ad419223a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:52:23.231241 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9a062d-e186-4347-9efe-e06ad419223a admin admin] REQUEST_METHOD: `PUT` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:23.231517 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9a062d-e186-4347-9efe-e06ad419223a admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:23.231787 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9a062d-e186-4347-9efe-e06ad419223a admin admin] PATH_INFO: `/v3/projects/e0c3e0d334a5419482afd7965d37c37a/users/0c0c48c8fb1d4c14adf429c93f6aab9b/roles/49e47cd157d743f6905f1db07ec878a7` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:23.248029 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9a062d-e186-4347-9efe-e06ad419223a admin admin] RBAC: Authorizing `identity:create_grant(project_id=e0c3e0d334a5419482afd7965d37c37a, user_id=0c0c48c8fb1d4c14adf429c93f6aab9b, role_id=49e47cd157d743f6905f1db07ec878a7)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:52:23.250621 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9a062d-e186-4347-9efe-e06ad419223a admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:52:23.328122 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1306/2612] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Aug 24 06:52:23 2022] PUT /identity/v3/projects/e0c3e0d334a5419482afd7965d37c37a/users/0c0c48c8fb1d4c14adf429c93f6aab9b/roles/49e47cd157d743f6905f1db07ec878a7 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:52:23.332393 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266fd535-d7ac-4fa9-b15f-7710ea98d726 admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:52:23.342356 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266fd535-d7ac-4fa9-b15f-7710ea98d726 admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:52:23.399236 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97bdf35-269e-4a27-b290-deb743286c07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:52:23.399848 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97bdf35-269e-4a27-b290-deb743286c07 admin admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:23.400126 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97bdf35-269e-4a27-b290-deb743286c07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:23.400394 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97bdf35-269e-4a27-b290-deb743286c07 admin admin] PATH_INFO: `/v3/roles` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:23.401066 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97bdf35-269e-4a27-b290-deb743286c07 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:52:23.402654 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97bdf35-269e-4a27-b290-deb743286c07 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:52:23.412718 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1307/2613] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Aug 24 06:52:23 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:52:23.417463 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9a062d-e186-4347-9efe-e06ad419223a admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:52:23.427626 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9a062d-e186-4347-9efe-e06ad419223a admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:52:23.432891 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a4f66c-6969-4d3c-bb8c-4dc1d2d9846a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:52:23.433536 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a4f66c-6969-4d3c-bb8c-4dc1d2d9846a admin admin] REQUEST_METHOD: `PUT` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:23.433807 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a4f66c-6969-4d3c-bb8c-4dc1d2d9846a admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:23.434068 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a4f66c-6969-4d3c-bb8c-4dc1d2d9846a admin admin] PATH_INFO: `/v3/projects/e0c3e0d334a5419482afd7965d37c37a/users/0c0c48c8fb1d4c14adf429c93f6aab9b/roles/49e47cd157d743f6905f1db07ec878a7` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:23.436239 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a4f66c-6969-4d3c-bb8c-4dc1d2d9846a admin admin] RBAC: Authorizing `identity:create_grant(project_id=e0c3e0d334a5419482afd7965d37c37a, user_id=0c0c48c8fb1d4c14adf429c93f6aab9b, role_id=49e47cd157d743f6905f1db07ec878a7)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:52:23.438861 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a4f66c-6969-4d3c-bb8c-4dc1d2d9846a admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:52:23.449051 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1307/2614] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Aug 24 06:52:23 2022] PUT /identity/v3/projects/e0c3e0d334a5419482afd7965d37c37a/users/0c0c48c8fb1d4c14adf429c93f6aab9b/roles/49e47cd157d743f6905f1db07ec878a7 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:52:23.460572 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a819212e-2f70-4be3-99e5-65440542fdc7 None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:23.460865 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a819212e-2f70-4be3-99e5-65440542fdc7 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:23.461139 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a819212e-2f70-4be3-99e5-65440542fdc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:23.489019 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-a819212e-2f70-4be3-99e5-65440542fdc7 None None] MFA Rules not processed for user `0c0c48c8fb1d4c14adf429c93f6aab9b`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:52:23.490225 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-a819212e-2f70-4be3-99e5-65440542fdc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:52:23.559244 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1308/2615] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:52:23 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:52:23.570797 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a4f66c-6969-4d3c-bb8c-4dc1d2d9846a admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:52:23.572514 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-b6a4f66c-6969-4d3c-bb8c-4dc1d2d9846a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:52:23.583645 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:52:23.633543 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a4f66c-6969-4d3c-bb8c-4dc1d2d9846a admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:52:23.637356 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d3ae0d-edc5-45b8-b6b2-8de6c24cf516 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c4def55ac0e48b29836bc5c64bc7402', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:52:23.637966 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d3ae0d-edc5-45b8-b6b2-8de6c24cf516 service cinder] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:23.638246 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d3ae0d-edc5-45b8-b6b2-8de6c24cf516 service cinder] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:23.638519 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d3ae0d-edc5-45b8-b6b2-8de6c24cf516 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:23.700919 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d3ae0d-edc5-45b8-b6b2-8de6c24cf516 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:52:23.702435 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d3ae0d-edc5-45b8-b6b2-8de6c24cf516 service cinder] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:52:23.712544 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1308/2616] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:52:23 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:52:27.877292 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a819212e-2f70-4be3-99e5-65440542fdc7 None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:52:27.931205 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:52:27.946248 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a819212e-2f70-4be3-99e5-65440542fdc7 None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:52:27.951027 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab266f3-8ce7-445e-85a1-3243470bb731 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:52:27.951710 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab266f3-8ce7-445e-85a1-3243470bb731 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:27.952011 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab266f3-8ce7-445e-85a1-3243470bb731 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:27.952290 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab266f3-8ce7-445e-85a1-3243470bb731 admin admin] PATH_INFO: `/v3/users/ea128ec658964a1ba7a3ada90359c847` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:27.953455 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab266f3-8ce7-445e-85a1-3243470bb731 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ea128ec658964a1ba7a3ada90359c847)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:52:27.955406 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab266f3-8ce7-445e-85a1-3243470bb731 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:52:28.173501 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-6ab266f3-8ce7-445e-85a1-3243470bb731 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ea128ec658964a1ba7a3ada90359c847'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:52:28.180489 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-6ab266f3-8ce7-445e-85a1-3243470bb731 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ea128ec658964a1ba7a3ada90359c847'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:52:28.181200 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-6ab266f3-8ce7-445e-85a1-3243470bb731 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ea128ec658964a1ba7a3ada90359c847'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:52:28.181804 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-6ab266f3-8ce7-445e-85a1-3243470bb731 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ea128ec658964a1ba7a3ada90359c847'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:52:28.284557 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-6ab266f3-8ce7-445e-85a1-3243470bb731 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ea128ec658964a1ba7a3ada90359c847'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:52:28.305247 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1309/2617] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Aug 24 06:52:27 2022] DELETE /identity/v3/users/ea128ec658964a1ba7a3ada90359c847 => generated 0 bytes in 428 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:52:28.309895 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ccf6b8-0db1-4a78-8ec9-ef00527bd1c9 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:28.310705 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ccf6b8-0db1-4a78-8ec9-ef00527bd1c9 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:28.310705 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ccf6b8-0db1-4a78-8ec9-ef00527bd1c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:28.330356 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-f4ccf6b8-0db1-4a78-8ec9-ef00527bd1c9 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:52:28.331597 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-f4ccf6b8-0db1-4a78-8ec9-ef00527bd1c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:52:28.393084 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1309/2618] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:52:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 06:52:28.403408 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab266f3-8ce7-445e-85a1-3243470bb731 admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:52:28.404855 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-6ab266f3-8ce7-445e-85a1-3243470bb731 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:52:28.467718 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab266f3-8ce7-445e-85a1-3243470bb731 admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:52:28.471552 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d049d1-653f-4c24-9611-f325f80e9bce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bb6c79db75e4d169331c02133d1ec32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:52:28.472175 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d049d1-653f-4c24-9611-f325f80e9bce service neutron] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:28.472458 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d049d1-653f-4c24-9611-f325f80e9bce service neutron] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:28.472772 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d049d1-653f-4c24-9611-f325f80e9bce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:28.483004 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d049d1-653f-4c24-9611-f325f80e9bce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:52:28.484453 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d049d1-653f-4c24-9611-f325f80e9bce service neutron] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:52:28.495561 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1310/2619] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:52:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:52:28.946370 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ccf6b8-0db1-4a78-8ec9-ef00527bd1c9 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:52:28.948456 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-f4ccf6b8-0db1-4a78-8ec9-ef00527bd1c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:52:28.965484 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ccf6b8-0db1-4a78-8ec9-ef00527bd1c9 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:52:28.980004 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84edc61-73af-4743-9ef5-b4a91aab37b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:52:28.980739 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84edc61-73af-4743-9ef5-b4a91aab37b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:28.983512 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84edc61-73af-4743-9ef5-b4a91aab37b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:28.983512 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84edc61-73af-4743-9ef5-b4a91aab37b6 admin admin] PATH_INFO: `/v3/projects/2a3a7c908b22404ba3159609cb5c7c69` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:28.983512 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84edc61-73af-4743-9ef5-b4a91aab37b6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2a3a7c908b22404ba3159609cb5c7c69)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:52:28.984550 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84edc61-73af-4743-9ef5-b4a91aab37b6 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:52:29.134018 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-b84edc61-73af-4743-9ef5-b4a91aab37b6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2a3a7c908b22404ba3159609cb5c7c69', 'request_id': 'req-b84edc61-73af-4743-9ef5-b4a91aab37b6'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:52:29.140748 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-b84edc61-73af-4743-9ef5-b4a91aab37b6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2a3a7c908b22404ba3159609cb5c7c69', 'request_id': 'req-b84edc61-73af-4743-9ef5-b4a91aab37b6'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:52:29.142045 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:52:29.148018 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-b84edc61-73af-4743-9ef5-b4a91aab37b6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2a3a7c908b22404ba3159609cb5c7c69', 'request_id': 'req-b84edc61-73af-4743-9ef5-b4a91aab37b6'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:52:29.235849 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-b84edc61-73af-4743-9ef5-b4a91aab37b6 admin admin] The token cache is being invalidate because project 2a3a7c908b22404ba3159609cb5c7c69 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81852) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:52:29.236064 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-b84edc61-73af-4743-9ef5-b4a91aab37b6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:52:29.236746 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-b84edc61-73af-4743-9ef5-b4a91aab37b6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:52:29.238239 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1310/2620] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Aug 24 06:52:28 2022] DELETE /identity/v3/projects/2a3a7c908b22404ba3159609cb5c7c69 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:52:29.242288 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d049d1-653f-4c24-9611-f325f80e9bce service neutron] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:52:29.253147 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d049d1-653f-4c24-9611-f325f80e9bce service neutron] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:52:29.309923 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6778fdbc-9940-4594-baed-1aec27a965c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:52:29.310921 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6778fdbc-9940-4594-baed-1aec27a965c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:29.311199 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6778fdbc-9940-4594-baed-1aec27a965c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:29.311468 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6778fdbc-9940-4594-baed-1aec27a965c9 admin admin] PATH_INFO: `/v3/users/0c0c48c8fb1d4c14adf429c93f6aab9b` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:29.312601 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6778fdbc-9940-4594-baed-1aec27a965c9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0c0c48c8fb1d4c14adf429c93f6aab9b)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:52:29.314518 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6778fdbc-9940-4594-baed-1aec27a965c9 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:52:29.327218 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:52:29.515759 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-6778fdbc-9940-4594-baed-1aec27a965c9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0c0c48c8fb1d4c14adf429c93f6aab9b'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:52:29.522812 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-6778fdbc-9940-4594-baed-1aec27a965c9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0c0c48c8fb1d4c14adf429c93f6aab9b'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:52:29.529024 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-6778fdbc-9940-4594-baed-1aec27a965c9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0c0c48c8fb1d4c14adf429c93f6aab9b'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:52:29.529024 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-6778fdbc-9940-4594-baed-1aec27a965c9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0c0c48c8fb1d4c14adf429c93f6aab9b'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:52:29.613477 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-6778fdbc-9940-4594-baed-1aec27a965c9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0c0c48c8fb1d4c14adf429c93f6aab9b'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:52:29.631745 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1311/2621] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Aug 24 06:52:29 2022] DELETE /identity/v3/users/0c0c48c8fb1d4c14adf429c93f6aab9b => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:52:30.064293 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84edc61-73af-4743-9ef5-b4a91aab37b6 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:52:30.065813 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-b84edc61-73af-4743-9ef5-b4a91aab37b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:52:30.080270 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84edc61-73af-4743-9ef5-b4a91aab37b6 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:52:30.139237 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f5c6fe-a43d-49af-9638-5376b42155f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:52:30.139872 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f5c6fe-a43d-49af-9638-5376b42155f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:30.140144 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f5c6fe-a43d-49af-9638-5376b42155f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:30.140412 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f5c6fe-a43d-49af-9638-5376b42155f3 admin admin] PATH_INFO: `/v3/projects/e0c3e0d334a5419482afd7965d37c37a` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:30.141637 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f5c6fe-a43d-49af-9638-5376b42155f3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e0c3e0d334a5419482afd7965d37c37a)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:52:30.143545 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f5c6fe-a43d-49af-9638-5376b42155f3 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:52:30.280723 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-29f5c6fe-a43d-49af-9638-5376b42155f3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e0c3e0d334a5419482afd7965d37c37a', 'request_id': 'req-29f5c6fe-a43d-49af-9638-5376b42155f3'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:52:30.286947 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-29f5c6fe-a43d-49af-9638-5376b42155f3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e0c3e0d334a5419482afd7965d37c37a', 'request_id': 'req-29f5c6fe-a43d-49af-9638-5376b42155f3'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:52:30.293245 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-29f5c6fe-a43d-49af-9638-5376b42155f3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e0c3e0d334a5419482afd7965d37c37a', 'request_id': 'req-29f5c6fe-a43d-49af-9638-5376b42155f3'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:52:30.373552 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-29f5c6fe-a43d-49af-9638-5376b42155f3 admin admin] The token cache is being invalidate because project e0c3e0d334a5419482afd7965d37c37a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81852) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:52:30.373844 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-29f5c6fe-a43d-49af-9638-5376b42155f3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:52:30.374520 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-29f5c6fe-a43d-49af-9638-5376b42155f3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:52:30.376054 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1311/2622] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Aug 24 06:52:30 2022] DELETE /identity/v3/projects/e0c3e0d334a5419482afd7965d37c37a => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:52:39.628041 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=22,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:52:40.369772 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=55,keystone:INSERT=2,keystone:DELETE=8 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:52:40.561464 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6778fdbc-9940-4594-baed-1aec27a965c9 admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:52:40.562560 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-6778fdbc-9940-4594-baed-1aec27a965c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:52:40.633007 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6778fdbc-9940-4594-baed-1aec27a965c9 admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:52:40.681796 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba69385-2bb4-449b-8e53-869623ddcf09 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c4def55ac0e48b29836bc5c64bc7402', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:52:40.682402 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba69385-2bb4-449b-8e53-869623ddcf09 service cinder] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:40.682683 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba69385-2bb4-449b-8e53-869623ddcf09 service cinder] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:40.682959 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba69385-2bb4-449b-8e53-869623ddcf09 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:40.684782 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-bba69385-2bb4-449b-8e53-869623ddcf09 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:52:40.758734 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba69385-2bb4-449b-8e53-869623ddcf09 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:52:40.760740 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba69385-2bb4-449b-8e53-869623ddcf09 service cinder] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:52:40.769994 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1312/2623] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:52:40 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:52:44.143093 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f5c6fe-a43d-49af-9638-5376b42155f3 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:52:44.144519 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-29f5c6fe-a43d-49af-9638-5376b42155f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:52:44.210258 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f5c6fe-a43d-49af-9638-5376b42155f3 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:52:44.214939 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808da711-fd40-4517-99c2-e64b661273e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:52:44.215569 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808da711-fd40-4517-99c2-e64b661273e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:44.215845 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808da711-fd40-4517-99c2-e64b661273e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:44.216118 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808da711-fd40-4517-99c2-e64b661273e8 admin admin] PATH_INFO: `/v3/users/85450e77c8334a08a7d09a44df532ce5` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:44.217266 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808da711-fd40-4517-99c2-e64b661273e8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=85450e77c8334a08a7d09a44df532ce5)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:52:44.219196 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808da711-fd40-4517-99c2-e64b661273e8 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:52:44.427259 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-808da711-fd40-4517-99c2-e64b661273e8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '85450e77c8334a08a7d09a44df532ce5'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:52:44.434898 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-808da711-fd40-4517-99c2-e64b661273e8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '85450e77c8334a08a7d09a44df532ce5'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:52:44.435390 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-808da711-fd40-4517-99c2-e64b661273e8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '85450e77c8334a08a7d09a44df532ce5'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:52:44.504771 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-808da711-fd40-4517-99c2-e64b661273e8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '85450e77c8334a08a7d09a44df532ce5'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:52:44.517380 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-808da711-fd40-4517-99c2-e64b661273e8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '85450e77c8334a08a7d09a44df532ce5'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:52:44.523442 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1312/2624] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Aug 24 06:52:44 2022] DELETE /identity/v3/users/85450e77c8334a08a7d09a44df532ce5 => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:52:44.528663 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fe204b-bdb8-47a3-908f-6e62b4aaa2c4 None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:44.529145 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fe204b-bdb8-47a3-908f-6e62b4aaa2c4 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:44.529243 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fe204b-bdb8-47a3-908f-6e62b4aaa2c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:44.548577 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-d3fe204b-bdb8-47a3-908f-6e62b4aaa2c4 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:52:44.549774 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-d3fe204b-bdb8-47a3-908f-6e62b4aaa2c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:52:44.594769 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=100 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:52:44.611165 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1313/2625] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:52:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:52:44.620981 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808da711-fd40-4517-99c2-e64b661273e8 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:52:44.622407 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-808da711-fd40-4517-99c2-e64b661273e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:52:44.679522 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808da711-fd40-4517-99c2-e64b661273e8 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:52:44.683453 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbce44e7-440e-4910-b3b9-ed527df871f8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bb6c79db75e4d169331c02133d1ec32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:52:44.684062 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbce44e7-440e-4910-b3b9-ed527df871f8 service neutron] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:44.684393 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbce44e7-440e-4910-b3b9-ed527df871f8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:44.684708 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbce44e7-440e-4910-b3b9-ed527df871f8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:44.694631 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbce44e7-440e-4910-b3b9-ed527df871f8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:52:44.696592 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbce44e7-440e-4910-b3b9-ed527df871f8 service neutron] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:52:44.707114 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1313/2626] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:52:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:52:45.133491 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fe204b-bdb8-47a3-908f-6e62b4aaa2c4 None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:52:45.135451 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-d3fe204b-bdb8-47a3-908f-6e62b4aaa2c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:52:45.147725 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fe204b-bdb8-47a3-908f-6e62b4aaa2c4 None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:52:45.153469 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586b6348-b719-45b7-80e9-a28a7a00b9b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:52:45.154144 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586b6348-b719-45b7-80e9-a28a7a00b9b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:45.154423 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586b6348-b719-45b7-80e9-a28a7a00b9b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:45.154695 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586b6348-b719-45b7-80e9-a28a7a00b9b5 admin admin] PATH_INFO: `/v3/projects/a7e3687c114142e69f1812dbf2613c63` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:45.155883 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586b6348-b719-45b7-80e9-a28a7a00b9b5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a7e3687c114142e69f1812dbf2613c63)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:52:45.157858 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586b6348-b719-45b7-80e9-a28a7a00b9b5 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:52:45.265458 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-586b6348-b719-45b7-80e9-a28a7a00b9b5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a7e3687c114142e69f1812dbf2613c63', 'request_id': 'req-586b6348-b719-45b7-80e9-a28a7a00b9b5'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:52:45.270735 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-586b6348-b719-45b7-80e9-a28a7a00b9b5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a7e3687c114142e69f1812dbf2613c63', 'request_id': 'req-586b6348-b719-45b7-80e9-a28a7a00b9b5'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:52:45.277417 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-586b6348-b719-45b7-80e9-a28a7a00b9b5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a7e3687c114142e69f1812dbf2613c63', 'request_id': 'req-586b6348-b719-45b7-80e9-a28a7a00b9b5'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:52:45.338300 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-586b6348-b719-45b7-80e9-a28a7a00b9b5 admin admin] The token cache is being invalidate because project a7e3687c114142e69f1812dbf2613c63 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81851) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:52:45.338586 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-586b6348-b719-45b7-80e9-a28a7a00b9b5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:52:45.339262 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-586b6348-b719-45b7-80e9-a28a7a00b9b5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:52:45.340784 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1314/2627] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Aug 24 06:52:45 2022] DELETE /identity/v3/projects/a7e3687c114142e69f1812dbf2613c63 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:52:45.345728 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed8b816-ee46-47c6-8a4d-27b98866209a None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:45.346181 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed8b816-ee46-47c6-8a4d-27b98866209a None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:45.346263 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed8b816-ee46-47c6-8a4d-27b98866209a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:45.366889 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-1ed8b816-ee46-47c6-8a4d-27b98866209a None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:52:45.368085 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-1ed8b816-ee46-47c6-8a4d-27b98866209a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:52:45.385902 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:52:45.426970 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1314/2628] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:52:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:52:45.436393 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68633097-750e-4b28-82e4-a9dea910cee6 None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:45.436707 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68633097-750e-4b28-82e4-a9dea910cee6 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:45.436981 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68633097-750e-4b28-82e4-a9dea910cee6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:45.458477 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-68633097-750e-4b28-82e4-a9dea910cee6 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:52:45.459646 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-68633097-750e-4b28-82e4-a9dea910cee6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:52:45.467638 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1315/2629] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:52:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 06:52:45.472050 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed8b816-ee46-47c6-8a4d-27b98866209a None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:52:45.482079 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed8b816-ee46-47c6-8a4d-27b98866209a None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:52:45.487859 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee31b1a-e7b7-49a2-b5f3-7b95ef3a8ad4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:52:45.488478 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee31b1a-e7b7-49a2-b5f3-7b95ef3a8ad4 admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:45.488797 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee31b1a-e7b7-49a2-b5f3-7b95ef3a8ad4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:45.489247 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee31b1a-e7b7-49a2-b5f3-7b95ef3a8ad4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:45.490240 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee31b1a-e7b7-49a2-b5f3-7b95ef3a8ad4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:52:45.490547 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee31b1a-e7b7-49a2-b5f3-7b95ef3a8ad4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:52:45.492154 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee31b1a-e7b7-49a2-b5f3-7b95ef3a8ad4 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:52:45.504558 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1315/2630] 10.222.0.62 () {58 vars in 1153 bytes} [Wed Aug 24 06:52:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:52:45.509119 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68633097-750e-4b28-82e4-a9dea910cee6 None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:52:45.519424 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68633097-750e-4b28-82e4-a9dea910cee6 None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:52:45.524184 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e514b920-24cf-4805-a7cf-73a185fdc92b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:52:45.524829 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e514b920-24cf-4805-a7cf-73a185fdc92b admin admin] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:45.525109 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e514b920-24cf-4805-a7cf-73a185fdc92b admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:45.525379 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e514b920-24cf-4805-a7cf-73a185fdc92b admin admin] PATH_INFO: `/v3/projects` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:45.526000 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e514b920-24cf-4805-a7cf-73a185fdc92b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:52:45.527918 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e514b920-24cf-4805-a7cf-73a185fdc92b admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:52:45.638635 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1316/2631] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Aug 24 06:52:45 2022] POST /identity/v3/projects => generated 383 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:52:45.643105 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee31b1a-e7b7-49a2-b5f3-7b95ef3a8ad4 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:52:45.653438 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee31b1a-e7b7-49a2-b5f3-7b95ef3a8ad4 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:52:45.713505 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4684480c-6900-45c0-9ed5-9c9ea38995cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:52:45.714118 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4684480c-6900-45c0-9ed5-9c9ea38995cd admin admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:45.714396 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4684480c-6900-45c0-9ed5-9c9ea38995cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:45.714665 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4684480c-6900-45c0-9ed5-9c9ea38995cd admin admin] PATH_INFO: `/v3/users` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:45.715286 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4684480c-6900-45c0-9ed5-9c9ea38995cd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:52:45.717248 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4684480c-6900-45c0-9ed5-9c9ea38995cd admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:52:45.814693 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1316/2632] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Aug 24 06:52:45 2022] POST /identity/v3/users => generated 292 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:52:45.816176 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e514b920-24cf-4805-a7cf-73a185fdc92b admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:52:45.827429 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e514b920-24cf-4805-a7cf-73a185fdc92b admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:52:45.832164 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc866c05-4455-4bdc-b6fa-9b3b77cc66f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:52:45.832832 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc866c05-4455-4bdc-b6fa-9b3b77cc66f7 admin admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:45.833103 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc866c05-4455-4bdc-b6fa-9b3b77cc66f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:45.833583 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc866c05-4455-4bdc-b6fa-9b3b77cc66f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:45.834236 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc866c05-4455-4bdc-b6fa-9b3b77cc66f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:52:45.835836 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc866c05-4455-4bdc-b6fa-9b3b77cc66f7 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:52:45.846357 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1317/2633] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Aug 24 06:52:45 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:52:45.850510 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4684480c-6900-45c0-9ed5-9c9ea38995cd admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:52:45.861228 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4684480c-6900-45c0-9ed5-9c9ea38995cd admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:52:45.866084 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e247e2a3-cba8-464a-a3ea-f001ae2add65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:52:45.866691 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e247e2a3-cba8-464a-a3ea-f001ae2add65 admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:45.866970 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e247e2a3-cba8-464a-a3ea-f001ae2add65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:45.867241 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e247e2a3-cba8-464a-a3ea-f001ae2add65 admin admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:45.867882 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e247e2a3-cba8-464a-a3ea-f001ae2add65 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:52:45.870112 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e247e2a3-cba8-464a-a3ea-f001ae2add65 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:52:45.885750 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1317/2634] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Aug 24 06:52:45 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:52:45.889871 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc866c05-4455-4bdc-b6fa-9b3b77cc66f7 admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:52:45.900210 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc866c05-4455-4bdc-b6fa-9b3b77cc66f7 admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:52:45.906101 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef75de1d-9859-4bde-bcd8-67db29aa543c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:52:45.906765 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef75de1d-9859-4bde-bcd8-67db29aa543c admin admin] REQUEST_METHOD: `PUT` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:45.907050 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef75de1d-9859-4bde-bcd8-67db29aa543c admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:45.907326 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef75de1d-9859-4bde-bcd8-67db29aa543c admin admin] PATH_INFO: `/v3/projects/5196bfb7bf5a4fa1b23a15bc5d643468/users/3b8ccf375d554c2e97611933363371ee/roles/580aa7dbb38d4cff8f9d1bb8983f99ef` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:45.923178 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef75de1d-9859-4bde-bcd8-67db29aa543c admin admin] RBAC: Authorizing `identity:create_grant(project_id=5196bfb7bf5a4fa1b23a15bc5d643468, user_id=3b8ccf375d554c2e97611933363371ee, role_id=580aa7dbb38d4cff8f9d1bb8983f99ef)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:52:45.925960 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef75de1d-9859-4bde-bcd8-67db29aa543c admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:52:46.028857 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1318/2635] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Aug 24 06:52:45 2022] PUT /identity/v3/projects/5196bfb7bf5a4fa1b23a15bc5d643468/users/3b8ccf375d554c2e97611933363371ee/roles/580aa7dbb38d4cff8f9d1bb8983f99ef => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:52:46.040600 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af27edf-9a48-4678-994a-04587b96a492 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:46.040985 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af27edf-9a48-4678-994a-04587b96a492 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:46.041737 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af27edf-9a48-4678-994a-04587b96a492 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:46.071949 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-5af27edf-9a48-4678-994a-04587b96a492 None None] MFA Rules not processed for user `3b8ccf375d554c2e97611933363371ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:52:46.073255 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-5af27edf-9a48-4678-994a-04587b96a492 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:52:46.141460 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1318/2636] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:52:46 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:52:46.146094 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef75de1d-9859-4bde-bcd8-67db29aa543c admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:52:46.208655 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef75de1d-9859-4bde-bcd8-67db29aa543c admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:52:46.213903 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19eabeda-beab-4694-b5f4-8cfc2a7fa155 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:52:46.214563 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19eabeda-beab-4694-b5f4-8cfc2a7fa155 admin admin] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:46.214844 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19eabeda-beab-4694-b5f4-8cfc2a7fa155 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:46.215113 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19eabeda-beab-4694-b5f4-8cfc2a7fa155 admin admin] PATH_INFO: `/v3/projects` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:46.215726 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19eabeda-beab-4694-b5f4-8cfc2a7fa155 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:52:46.217647 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19eabeda-beab-4694-b5f4-8cfc2a7fa155 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:52:46.232989 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:52:46.338335 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1319/2637] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Aug 24 06:52:46 2022] POST /identity/v3/projects => generated 385 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:52:46.342694 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af27edf-9a48-4678-994a-04587b96a492 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:52:46.355633 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af27edf-9a48-4678-994a-04587b96a492 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:52:46.368073 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:52:46.411116 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97eee62b-a79b-425c-b7d0-03e62a5b5ba5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:52:46.411726 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97eee62b-a79b-425c-b7d0-03e62a5b5ba5 admin admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:46.412004 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97eee62b-a79b-425c-b7d0-03e62a5b5ba5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:46.412272 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97eee62b-a79b-425c-b7d0-03e62a5b5ba5 admin admin] PATH_INFO: `/v3/users` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:46.412922 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97eee62b-a79b-425c-b7d0-03e62a5b5ba5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:52:46.414953 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97eee62b-a79b-425c-b7d0-03e62a5b5ba5 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:52:46.499752 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1319/2638] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Aug 24 06:52:46 2022] POST /identity/v3/users => generated 293 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:52:46.503834 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19eabeda-beab-4694-b5f4-8cfc2a7fa155 admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:52:46.515055 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19eabeda-beab-4694-b5f4-8cfc2a7fa155 admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:52:46.520487 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6fa25d-66c5-47bd-8285-c5ed08b2a613 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:52:46.521153 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6fa25d-66c5-47bd-8285-c5ed08b2a613 admin admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:46.521432 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6fa25d-66c5-47bd-8285-c5ed08b2a613 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:46.521704 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6fa25d-66c5-47bd-8285-c5ed08b2a613 admin admin] PATH_INFO: `/v3/roles` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:46.522345 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6fa25d-66c5-47bd-8285-c5ed08b2a613 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:52:46.523978 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6fa25d-66c5-47bd-8285-c5ed08b2a613 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:52:46.536048 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1320/2639] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Aug 24 06:52:46 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:52:46.540118 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97eee62b-a79b-425c-b7d0-03e62a5b5ba5 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:52:46.550238 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97eee62b-a79b-425c-b7d0-03e62a5b5ba5 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:52:46.554733 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecffe970-892e-4212-b1e8-873e01175696 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:52:46.555321 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecffe970-892e-4212-b1e8-873e01175696 admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:46.556563 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecffe970-892e-4212-b1e8-873e01175696 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:46.556563 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecffe970-892e-4212-b1e8-873e01175696 admin admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:46.556563 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecffe970-892e-4212-b1e8-873e01175696 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:52:46.558053 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecffe970-892e-4212-b1e8-873e01175696 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:52:46.573314 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1320/2640] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Aug 24 06:52:46 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:52:46.577819 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6fa25d-66c5-47bd-8285-c5ed08b2a613 admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:52:46.590919 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6fa25d-66c5-47bd-8285-c5ed08b2a613 admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:52:46.594195 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bde041-c811-4c0e-895b-3e7708565d64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:52:46.594926 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bde041-c811-4c0e-895b-3e7708565d64 admin admin] REQUEST_METHOD: `PUT` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:46.595219 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bde041-c811-4c0e-895b-3e7708565d64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:46.595493 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bde041-c811-4c0e-895b-3e7708565d64 admin admin] PATH_INFO: `/v3/projects/2501d568bc05414788a079db2fe07dae/users/cf68a58cc5c74ee28dc0fdb60bde3c52/roles/580aa7dbb38d4cff8f9d1bb8983f99ef` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:46.612953 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73bde041-c811-4c0e-895b-3e7708565d64 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2501d568bc05414788a079db2fe07dae, user_id=cf68a58cc5c74ee28dc0fdb60bde3c52, role_id=580aa7dbb38d4cff8f9d1bb8983f99ef)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:52:46.615871 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73bde041-c811-4c0e-895b-3e7708565d64 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:52:46.725955 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1321/2641] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Aug 24 06:52:46 2022] PUT /identity/v3/projects/2501d568bc05414788a079db2fe07dae/users/cf68a58cc5c74ee28dc0fdb60bde3c52/roles/580aa7dbb38d4cff8f9d1bb8983f99ef => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:52:46.737135 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddef529c-676e-426a-b0a6-99584219065c None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:46.737526 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddef529c-676e-426a-b0a6-99584219065c None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:46.737670 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddef529c-676e-426a-b0a6-99584219065c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:46.771081 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-ddef529c-676e-426a-b0a6-99584219065c None None] MFA Rules not processed for user `cf68a58cc5c74ee28dc0fdb60bde3c52`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:52:46.772330 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-ddef529c-676e-426a-b0a6-99584219065c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:52:46.858165 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1321/2642] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:52:46 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:52:46.863030 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bde041-c811-4c0e-895b-3e7708565d64 admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:52:46.928908 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bde041-c811-4c0e-895b-3e7708565d64 admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:52:46.934594 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d806b7a3-8aa1-4d9d-800e-c6222a711856 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:52:46.935437 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d806b7a3-8aa1-4d9d-800e-c6222a711856 admin admin] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:46.935974 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d806b7a3-8aa1-4d9d-800e-c6222a711856 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:46.936245 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d806b7a3-8aa1-4d9d-800e-c6222a711856 admin admin] PATH_INFO: `/v3/projects` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:46.936938 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d806b7a3-8aa1-4d9d-800e-c6222a711856 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:52:46.938849 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d806b7a3-8aa1-4d9d-800e-c6222a711856 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:52:47.051375 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1322/2643] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Aug 24 06:52:46 2022] POST /identity/v3/projects => generated 381 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:52:47.056088 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddef529c-676e-426a-b0a6-99584219065c None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:52:47.067679 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddef529c-676e-426a-b0a6-99584219065c None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:52:47.147838 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3317162-045a-474e-8fad-80336f075b59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:52:47.148461 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3317162-045a-474e-8fad-80336f075b59 admin admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:47.148761 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3317162-045a-474e-8fad-80336f075b59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:47.149031 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3317162-045a-474e-8fad-80336f075b59 admin admin] PATH_INFO: `/v3/users` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:47.149673 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3317162-045a-474e-8fad-80336f075b59 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:52:47.151641 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3317162-045a-474e-8fad-80336f075b59 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:52:47.240528 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1322/2644] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Aug 24 06:52:47 2022] POST /identity/v3/users => generated 297 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:52:47.245243 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d806b7a3-8aa1-4d9d-800e-c6222a711856 admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:52:47.256349 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d806b7a3-8aa1-4d9d-800e-c6222a711856 admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:52:47.261979 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a084e9c-cb31-43ea-b8e0-f83a0dada60c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:52:47.262734 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a084e9c-cb31-43ea-b8e0-f83a0dada60c admin admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:47.263021 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a084e9c-cb31-43ea-b8e0-f83a0dada60c admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:47.263383 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a084e9c-cb31-43ea-b8e0-f83a0dada60c admin admin] PATH_INFO: `/v3/roles` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:47.264341 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a084e9c-cb31-43ea-b8e0-f83a0dada60c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:52:47.266545 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a084e9c-cb31-43ea-b8e0-f83a0dada60c admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:52:47.279925 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1323/2645] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Aug 24 06:52:47 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:52:47.284247 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3317162-045a-474e-8fad-80336f075b59 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:52:47.295565 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3317162-045a-474e-8fad-80336f075b59 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:52:47.300300 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46de502d-53da-4019-9cec-0221cfbe7e3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:52:47.301005 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46de502d-53da-4019-9cec-0221cfbe7e3b admin admin] REQUEST_METHOD: `PUT` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:47.301287 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46de502d-53da-4019-9cec-0221cfbe7e3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:47.303572 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46de502d-53da-4019-9cec-0221cfbe7e3b admin admin] PATH_INFO: `/v3/projects/da81e6e65f604295968d2b65c5f86b64/users/9bd5706cbb40450c9460bebc34ac3172/roles/49e47cd157d743f6905f1db07ec878a7` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:47.315205 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:52:47.318472 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46de502d-53da-4019-9cec-0221cfbe7e3b admin admin] RBAC: Authorizing `identity:create_grant(project_id=da81e6e65f604295968d2b65c5f86b64, user_id=9bd5706cbb40450c9460bebc34ac3172, role_id=49e47cd157d743f6905f1db07ec878a7)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:52:47.321263 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46de502d-53da-4019-9cec-0221cfbe7e3b admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:52:47.423119 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1323/2646] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Aug 24 06:52:47 2022] PUT /identity/v3/projects/da81e6e65f604295968d2b65c5f86b64/users/9bd5706cbb40450c9460bebc34ac3172/roles/49e47cd157d743f6905f1db07ec878a7 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 06:52:47.428725 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a084e9c-cb31-43ea-b8e0-f83a0dada60c admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:52:47.440563 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a084e9c-cb31-43ea-b8e0-f83a0dada60c admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:52:47.503276 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cc80ea-27c9-4653-8aec-62254512b163 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:52:47.503881 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cc80ea-27c9-4653-8aec-62254512b163 admin admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:47.504154 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cc80ea-27c9-4653-8aec-62254512b163 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:47.504425 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cc80ea-27c9-4653-8aec-62254512b163 admin admin] PATH_INFO: `/v3/roles` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:47.505085 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1cc80ea-27c9-4653-8aec-62254512b163 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:52:47.506682 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1cc80ea-27c9-4653-8aec-62254512b163 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:52:47.517454 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1324/2647] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Aug 24 06:52:47 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:52:47.521869 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46de502d-53da-4019-9cec-0221cfbe7e3b admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:52:47.533564 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46de502d-53da-4019-9cec-0221cfbe7e3b admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:52:47.538619 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3ae4d0-a35a-498e-9b49-b8108cbfbe4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:52:47.539291 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3ae4d0-a35a-498e-9b49-b8108cbfbe4c admin admin] REQUEST_METHOD: `PUT` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:47.539573 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3ae4d0-a35a-498e-9b49-b8108cbfbe4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:47.539851 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3ae4d0-a35a-498e-9b49-b8108cbfbe4c admin admin] PATH_INFO: `/v3/projects/da81e6e65f604295968d2b65c5f86b64/users/9bd5706cbb40450c9460bebc34ac3172/roles/49e47cd157d743f6905f1db07ec878a7` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:47.542439 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3ae4d0-a35a-498e-9b49-b8108cbfbe4c admin admin] RBAC: Authorizing `identity:create_grant(project_id=da81e6e65f604295968d2b65c5f86b64, user_id=9bd5706cbb40450c9460bebc34ac3172, role_id=49e47cd157d743f6905f1db07ec878a7)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:52:47.545298 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3ae4d0-a35a-498e-9b49-b8108cbfbe4c admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:52:47.556815 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1324/2648] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Aug 24 06:52:47 2022] PUT /identity/v3/projects/da81e6e65f604295968d2b65c5f86b64/users/9bd5706cbb40450c9460bebc34ac3172/roles/49e47cd157d743f6905f1db07ec878a7 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:52:47.577583 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fdfce5-6225-48a3-bbf7-0b3cb487eebe None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:47.577869 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fdfce5-6225-48a3-bbf7-0b3cb487eebe None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:47.578386 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fdfce5-6225-48a3-bbf7-0b3cb487eebe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:47.613702 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-61fdfce5-6225-48a3-bbf7-0b3cb487eebe None None] MFA Rules not processed for user `9bd5706cbb40450c9460bebc34ac3172`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:52:47.614930 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-61fdfce5-6225-48a3-bbf7-0b3cb487eebe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:52:47.632523 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:52:47.695382 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1325/2649] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:52:47 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:52:47.710923 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3ae4d0-a35a-498e-9b49-b8108cbfbe4c admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:52:47.713252 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-3f3ae4d0-a35a-498e-9b49-b8108cbfbe4c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:52:47.779391 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3ae4d0-a35a-498e-9b49-b8108cbfbe4c admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:52:47.783265 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355f7067-75f9-4cae-bc6d-ef753bf273be service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c4def55ac0e48b29836bc5c64bc7402', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:52:47.783874 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355f7067-75f9-4cae-bc6d-ef753bf273be service cinder] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:47.784151 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355f7067-75f9-4cae-bc6d-ef753bf273be service cinder] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:47.784425 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355f7067-75f9-4cae-bc6d-ef753bf273be service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:47.842794 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355f7067-75f9-4cae-bc6d-ef753bf273be service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:52:47.844232 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355f7067-75f9-4cae-bc6d-ef753bf273be service cinder] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:52:47.854099 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1325/2650] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:52:47 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:52:49.836142 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fdfce5-6225-48a3-bbf7-0b3cb487eebe None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:52:49.837621 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-61fdfce5-6225-48a3-bbf7-0b3cb487eebe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:52:49.850150 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fdfce5-6225-48a3-bbf7-0b3cb487eebe None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:52:49.854310 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cba2b2-09dd-44ab-817f-9e1abfc7d78b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c4def55ac0e48b29836bc5c64bc7402', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:52:49.854924 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cba2b2-09dd-44ab-817f-9e1abfc7d78b service cinder] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:49.855198 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cba2b2-09dd-44ab-817f-9e1abfc7d78b service cinder] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:49.855473 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cba2b2-09dd-44ab-817f-9e1abfc7d78b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:49.918271 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6cba2b2-09dd-44ab-817f-9e1abfc7d78b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:52:49.919724 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6cba2b2-09dd-44ab-817f-9e1abfc7d78b service cinder] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:52:49.929181 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1326/2651] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:52:49 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:52:51.287215 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355f7067-75f9-4cae-bc6d-ef753bf273be service cinder] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:52:51.296395 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355f7067-75f9-4cae-bc6d-ef753bf273be service cinder] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:52:51.300126 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a6a4b2-8371-41d1-a981-17a4d73ff35e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c4def55ac0e48b29836bc5c64bc7402', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:52:51.300762 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a6a4b2-8371-41d1-a981-17a4d73ff35e service cinder] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:51.301040 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a6a4b2-8371-41d1-a981-17a4d73ff35e service cinder] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:51.301311 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a6a4b2-8371-41d1-a981-17a4d73ff35e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:51.310946 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a6a4b2-8371-41d1-a981-17a4d73ff35e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:52:51.312314 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a6a4b2-8371-41d1-a981-17a4d73ff35e service cinder] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:52:51.322570 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1326/2652] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:52:51 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:52:56.176363 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cba2b2-09dd-44ab-817f-9e1abfc7d78b service cinder] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:52:56.238756 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cba2b2-09dd-44ab-817f-9e1abfc7d78b service cinder] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:52:56.243227 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1dc0ee0-5f97-447c-9381-5ac59ec2a0d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:52:56.243849 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dc0ee0-5f97-447c-9381-5ac59ec2a0d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:56.244160 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dc0ee0-5f97-447c-9381-5ac59ec2a0d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:56.244418 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dc0ee0-5f97-447c-9381-5ac59ec2a0d5 admin admin] PATH_INFO: `/v3/users/3b8ccf375d554c2e97611933363371ee` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:56.245572 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1dc0ee0-5f97-447c-9381-5ac59ec2a0d5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3b8ccf375d554c2e97611933363371ee)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:52:56.247487 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1dc0ee0-5f97-447c-9381-5ac59ec2a0d5 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:52:56.441432 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-c1dc0ee0-5f97-447c-9381-5ac59ec2a0d5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3b8ccf375d554c2e97611933363371ee'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:52:56.446742 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:52:56.449166 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-c1dc0ee0-5f97-447c-9381-5ac59ec2a0d5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3b8ccf375d554c2e97611933363371ee'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:52:56.449855 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-c1dc0ee0-5f97-447c-9381-5ac59ec2a0d5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3b8ccf375d554c2e97611933363371ee'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:52:56.450469 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-c1dc0ee0-5f97-447c-9381-5ac59ec2a0d5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3b8ccf375d554c2e97611933363371ee'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:52:56.547674 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-c1dc0ee0-5f97-447c-9381-5ac59ec2a0d5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3b8ccf375d554c2e97611933363371ee'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:52:56.568469 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1327/2653] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Aug 24 06:52:56 2022] DELETE /identity/v3/users/3b8ccf375d554c2e97611933363371ee => generated 0 bytes in 393 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:52:56.573536 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487e55d6-826c-4909-a03a-ff50d13d98c3 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:56.574023 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487e55d6-826c-4909-a03a-ff50d13d98c3 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:56.574129 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487e55d6-826c-4909-a03a-ff50d13d98c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:56.595234 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-487e55d6-826c-4909-a03a-ff50d13d98c3 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:52:56.599870 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-487e55d6-826c-4909-a03a-ff50d13d98c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:52:56.662437 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1327/2654] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:52:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:52:56.672518 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1dc0ee0-5f97-447c-9381-5ac59ec2a0d5 admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:52:56.673916 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-c1dc0ee0-5f97-447c-9381-5ac59ec2a0d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:52:56.736179 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1dc0ee0-5f97-447c-9381-5ac59ec2a0d5 admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:52:56.740053 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73798c3-4103-460a-9e5e-7180798c9c0b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bb6c79db75e4d169331c02133d1ec32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:52:56.740694 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73798c3-4103-460a-9e5e-7180798c9c0b service neutron] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:56.740970 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73798c3-4103-460a-9e5e-7180798c9c0b service neutron] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:56.741239 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73798c3-4103-460a-9e5e-7180798c9c0b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:56.751233 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73798c3-4103-460a-9e5e-7180798c9c0b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:52:56.752689 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73798c3-4103-460a-9e5e-7180798c9c0b service neutron] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:52:56.762729 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1328/2655] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:52:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:52:57.200941 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487e55d6-826c-4909-a03a-ff50d13d98c3 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:52:57.202367 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-487e55d6-826c-4909-a03a-ff50d13d98c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:52:57.215104 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487e55d6-826c-4909-a03a-ff50d13d98c3 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:52:57.220146 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7d92a3-6be2-4938-89e8-0e787fa19df4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:52:57.221660 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7d92a3-6be2-4938-89e8-0e787fa19df4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:57.222123 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7d92a3-6be2-4938-89e8-0e787fa19df4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:57.222398 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7d92a3-6be2-4938-89e8-0e787fa19df4 admin admin] PATH_INFO: `/v3/projects/5196bfb7bf5a4fa1b23a15bc5d643468` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:57.223688 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7d92a3-6be2-4938-89e8-0e787fa19df4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5196bfb7bf5a4fa1b23a15bc5d643468)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:52:57.226226 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7d92a3-6be2-4938-89e8-0e787fa19df4 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:52:57.246787 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:52:57.412442 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-8f7d92a3-6be2-4938-89e8-0e787fa19df4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5196bfb7bf5a4fa1b23a15bc5d643468', 'request_id': 'req-8f7d92a3-6be2-4938-89e8-0e787fa19df4'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:52:57.418547 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-8f7d92a3-6be2-4938-89e8-0e787fa19df4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5196bfb7bf5a4fa1b23a15bc5d643468', 'request_id': 'req-8f7d92a3-6be2-4938-89e8-0e787fa19df4'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:52:57.425410 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-8f7d92a3-6be2-4938-89e8-0e787fa19df4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5196bfb7bf5a4fa1b23a15bc5d643468', 'request_id': 'req-8f7d92a3-6be2-4938-89e8-0e787fa19df4'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:52:57.558780 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-8f7d92a3-6be2-4938-89e8-0e787fa19df4 admin admin] The token cache is being invalidate because project 5196bfb7bf5a4fa1b23a15bc5d643468 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81852) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:52:57.559003 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-8f7d92a3-6be2-4938-89e8-0e787fa19df4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:52:57.559635 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-8f7d92a3-6be2-4938-89e8-0e787fa19df4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:52:57.561480 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1328/2656] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Aug 24 06:52:57 2022] DELETE /identity/v3/projects/5196bfb7bf5a4fa1b23a15bc5d643468 => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 06:52:57.565504 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73798c3-4103-460a-9e5e-7180798c9c0b service neutron] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:52:57.575400 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73798c3-4103-460a-9e5e-7180798c9c0b service neutron] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:52:57.635503 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52545db2-9f6d-4c42-b454-8b90808b9ff2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:52:57.636305 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52545db2-9f6d-4c42-b454-8b90808b9ff2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:57.636417 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52545db2-9f6d-4c42-b454-8b90808b9ff2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:57.636745 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52545db2-9f6d-4c42-b454-8b90808b9ff2 admin admin] PATH_INFO: `/v3/users/cf68a58cc5c74ee28dc0fdb60bde3c52` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:57.637869 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52545db2-9f6d-4c42-b454-8b90808b9ff2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cf68a58cc5c74ee28dc0fdb60bde3c52)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:52:57.639890 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52545db2-9f6d-4c42-b454-8b90808b9ff2 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:52:57.896714 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-52545db2-9f6d-4c42-b454-8b90808b9ff2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cf68a58cc5c74ee28dc0fdb60bde3c52'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:52:57.904254 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-52545db2-9f6d-4c42-b454-8b90808b9ff2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cf68a58cc5c74ee28dc0fdb60bde3c52'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:52:57.904951 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-52545db2-9f6d-4c42-b454-8b90808b9ff2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cf68a58cc5c74ee28dc0fdb60bde3c52'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:52:57.905574 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-52545db2-9f6d-4c42-b454-8b90808b9ff2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cf68a58cc5c74ee28dc0fdb60bde3c52'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:52:57.970243 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-52545db2-9f6d-4c42-b454-8b90808b9ff2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cf68a58cc5c74ee28dc0fdb60bde3c52'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:52:57.981648 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=86,keystone:INSERT=2,keystone:DELETE=12 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:52:57.989458 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1329/2657] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Aug 24 06:52:57 2022] DELETE /identity/v3/users/cf68a58cc5c74ee28dc0fdb60bde3c52 => generated 0 bytes in 424 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:52:58.415520 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7d92a3-6be2-4938-89e8-0e787fa19df4 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:52:58.418900 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-8f7d92a3-6be2-4938-89e8-0e787fa19df4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:52:58.434170 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7d92a3-6be2-4938-89e8-0e787fa19df4 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:52:58.494616 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b2f41f-85b0-4ecd-9352-a32646e77806 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:52:58.498989 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b2f41f-85b0-4ecd-9352-a32646e77806 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:58.498989 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b2f41f-85b0-4ecd-9352-a32646e77806 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:58.498989 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b2f41f-85b0-4ecd-9352-a32646e77806 admin admin] PATH_INFO: `/v3/projects/2501d568bc05414788a079db2fe07dae` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:58.498989 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b2f41f-85b0-4ecd-9352-a32646e77806 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2501d568bc05414788a079db2fe07dae)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:52:58.498989 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b2f41f-85b0-4ecd-9352-a32646e77806 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:52:58.646507 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-29b2f41f-85b0-4ecd-9352-a32646e77806 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2501d568bc05414788a079db2fe07dae', 'request_id': 'req-29b2f41f-85b0-4ecd-9352-a32646e77806'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:52:58.652801 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-29b2f41f-85b0-4ecd-9352-a32646e77806 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2501d568bc05414788a079db2fe07dae', 'request_id': 'req-29b2f41f-85b0-4ecd-9352-a32646e77806'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:52:58.660099 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-29b2f41f-85b0-4ecd-9352-a32646e77806 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2501d568bc05414788a079db2fe07dae', 'request_id': 'req-29b2f41f-85b0-4ecd-9352-a32646e77806'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:52:58.732808 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-29b2f41f-85b0-4ecd-9352-a32646e77806 admin admin] The token cache is being invalidate because project 2501d568bc05414788a079db2fe07dae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81852) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:52:58.734116 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-29b2f41f-85b0-4ecd-9352-a32646e77806 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:52:58.734646 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-29b2f41f-85b0-4ecd-9352-a32646e77806 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:52:58.739677 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1329/2658] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Aug 24 06:52:58 2022] DELETE /identity/v3/projects/2501d568bc05414788a079db2fe07dae => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:52:58.740728 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52545db2-9f6d-4c42-b454-8b90808b9ff2 admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:52:58.742841 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-52545db2-9f6d-4c42-b454-8b90808b9ff2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:52:58.757443 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52545db2-9f6d-4c42-b454-8b90808b9ff2 admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:52:58.817431 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bf30eb-55b6-4c16-a18c-44d9803d4e8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:52:58.818063 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bf30eb-55b6-4c16-a18c-44d9803d4e8d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:58.818340 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bf30eb-55b6-4c16-a18c-44d9803d4e8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:58.818611 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bf30eb-55b6-4c16-a18c-44d9803d4e8d admin admin] PATH_INFO: `/v3/users/9bd5706cbb40450c9460bebc34ac3172` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:58.819713 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8bf30eb-55b6-4c16-a18c-44d9803d4e8d admin admin] RBAC: Authorizing `identity:delete_user(user_id=9bd5706cbb40450c9460bebc34ac3172)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:52:58.821740 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8bf30eb-55b6-4c16-a18c-44d9803d4e8d admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:52:59.007282 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-f8bf30eb-55b6-4c16-a18c-44d9803d4e8d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9bd5706cbb40450c9460bebc34ac3172'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:52:59.014544 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-f8bf30eb-55b6-4c16-a18c-44d9803d4e8d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9bd5706cbb40450c9460bebc34ac3172'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:52:59.015252 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-f8bf30eb-55b6-4c16-a18c-44d9803d4e8d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9bd5706cbb40450c9460bebc34ac3172'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:52:59.015864 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-f8bf30eb-55b6-4c16-a18c-44d9803d4e8d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9bd5706cbb40450c9460bebc34ac3172'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:52:59.107867 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-f8bf30eb-55b6-4c16-a18c-44d9803d4e8d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9bd5706cbb40450c9460bebc34ac3172'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:52:59.126290 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1330/2659] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Aug 24 06:52:58 2022] DELETE /identity/v3/users/9bd5706cbb40450c9460bebc34ac3172 => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:52:59.561486 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b2f41f-85b0-4ecd-9352-a32646e77806 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:52:59.562987 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-29b2f41f-85b0-4ecd-9352-a32646e77806 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:52:59.576465 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b2f41f-85b0-4ecd-9352-a32646e77806 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:52:59.608811 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:52:59.635179 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fc59f5-5d1c-4e69-a347-adf52bc80a8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:52:59.635817 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fc59f5-5d1c-4e69-a347-adf52bc80a8e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:59.636092 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fc59f5-5d1c-4e69-a347-adf52bc80a8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:59.636361 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fc59f5-5d1c-4e69-a347-adf52bc80a8e admin admin] PATH_INFO: `/v3/projects/da81e6e65f604295968d2b65c5f86b64` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:59.637497 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60fc59f5-5d1c-4e69-a347-adf52bc80a8e admin admin] RBAC: Authorizing `identity:delete_project(project_id=da81e6e65f604295968d2b65c5f86b64)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:52:59.639388 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60fc59f5-5d1c-4e69-a347-adf52bc80a8e admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:52:59.801745 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-60fc59f5-5d1c-4e69-a347-adf52bc80a8e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'da81e6e65f604295968d2b65c5f86b64', 'request_id': 'req-60fc59f5-5d1c-4e69-a347-adf52bc80a8e'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:52:59.811422 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-60fc59f5-5d1c-4e69-a347-adf52bc80a8e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'da81e6e65f604295968d2b65c5f86b64', 'request_id': 'req-60fc59f5-5d1c-4e69-a347-adf52bc80a8e'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:52:59.819051 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-60fc59f5-5d1c-4e69-a347-adf52bc80a8e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'da81e6e65f604295968d2b65c5f86b64', 'request_id': 'req-60fc59f5-5d1c-4e69-a347-adf52bc80a8e'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:52:59.937181 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-60fc59f5-5d1c-4e69-a347-adf52bc80a8e admin admin] The token cache is being invalidate because project da81e6e65f604295968d2b65c5f86b64 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81852) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:52:59.937482 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-60fc59f5-5d1c-4e69-a347-adf52bc80a8e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:52:59.938131 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-60fc59f5-5d1c-4e69-a347-adf52bc80a8e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:52:59.939651 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1330/2660] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Aug 24 06:52:59 2022] DELETE /identity/v3/projects/da81e6e65f604295968d2b65c5f86b64 => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:52:59.944896 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601b3df7-d1ea-4491-97f2-5b5eb405bf57 None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:59.945386 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601b3df7-d1ea-4491-97f2-5b5eb405bf57 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:59.945529 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601b3df7-d1ea-4491-97f2-5b5eb405bf57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:59.966428 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-601b3df7-d1ea-4491-97f2-5b5eb405bf57 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:52:59.969683 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-601b3df7-d1ea-4491-97f2-5b5eb405bf57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:53:00.027120 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1331/2661] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:52:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:53:00.035002 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016949be-ba08-4e9d-a1c0-3c86dab9891e None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:53:00.035282 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016949be-ba08-4e9d-a1c0-3c86dab9891e None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:53:00.035556 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016949be-ba08-4e9d-a1c0-3c86dab9891e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:53:00.057415 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-016949be-ba08-4e9d-a1c0-3c86dab9891e None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:53:00.058609 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-016949be-ba08-4e9d-a1c0-3c86dab9891e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:53:00.066727 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1331/2662] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:53:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:53:00.070897 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601b3df7-d1ea-4491-97f2-5b5eb405bf57 None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:53:00.081117 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601b3df7-d1ea-4491-97f2-5b5eb405bf57 None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:53:00.085630 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5724ae2d-e883-48db-a465-409c851edb8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:53:00.086227 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5724ae2d-e883-48db-a465-409c851edb8a admin admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:53:00.086489 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5724ae2d-e883-48db-a465-409c851edb8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:53:00.086756 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5724ae2d-e883-48db-a465-409c851edb8a admin admin] PATH_INFO: `/v3/domains` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:53:00.087386 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5724ae2d-e883-48db-a465-409c851edb8a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:53:00.088300 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5724ae2d-e883-48db-a465-409c851edb8a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:53:00.089962 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5724ae2d-e883-48db-a465-409c851edb8a admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:53:00.098735 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:53:00.102582 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1332/2663] 10.222.0.62 () {58 vars in 1153 bytes} [Wed Aug 24 06:53:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:53:00.106934 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016949be-ba08-4e9d-a1c0-3c86dab9891e None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:53:00.116253 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016949be-ba08-4e9d-a1c0-3c86dab9891e None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:53:00.120989 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8b5e8b-a16e-412b-9be2-157c0868309a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:53:00.121615 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8b5e8b-a16e-412b-9be2-157c0868309a admin admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:53:00.121888 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8b5e8b-a16e-412b-9be2-157c0868309a admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:53:00.122163 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8b5e8b-a16e-412b-9be2-157c0868309a admin admin] PATH_INFO: `/v3/projects` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:53:00.122787 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8b5e8b-a16e-412b-9be2-157c0868309a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:53:00.124815 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8b5e8b-a16e-412b-9be2-157c0868309a admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:53:00.216309 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1332/2664] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Aug 24 06:53:00 2022] POST /identity/v3/projects => generated 363 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:53:00.220460 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5724ae2d-e883-48db-a465-409c851edb8a admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:53:00.230570 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5724ae2d-e883-48db-a465-409c851edb8a admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:53:00.287626 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca88d96c-6171-4ff1-8c8c-67cf60523b02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:53:00.288227 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca88d96c-6171-4ff1-8c8c-67cf60523b02 admin admin] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:53:00.288525 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca88d96c-6171-4ff1-8c8c-67cf60523b02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:53:00.288815 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca88d96c-6171-4ff1-8c8c-67cf60523b02 admin admin] PATH_INFO: `/v3/users` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:53:00.289438 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca88d96c-6171-4ff1-8c8c-67cf60523b02 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:53:00.291384 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca88d96c-6171-4ff1-8c8c-67cf60523b02 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:53:00.374840 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1333/2665] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Aug 24 06:53:00 2022] POST /identity/v3/users => generated 282 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:53:00.378912 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8b5e8b-a16e-412b-9be2-157c0868309a admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:53:00.390146 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8b5e8b-a16e-412b-9be2-157c0868309a admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:53:00.394874 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72310fd9-739e-4bd5-8280-6a4c8c52564d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:53:00.395471 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72310fd9-739e-4bd5-8280-6a4c8c52564d admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:53:00.395752 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72310fd9-739e-4bd5-8280-6a4c8c52564d admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:53:00.396027 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72310fd9-739e-4bd5-8280-6a4c8c52564d admin admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:53:00.396701 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72310fd9-739e-4bd5-8280-6a4c8c52564d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:53:00.398354 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72310fd9-739e-4bd5-8280-6a4c8c52564d admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:53:00.408681 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1333/2666] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Aug 24 06:53:00 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:53:00.413201 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca88d96c-6171-4ff1-8c8c-67cf60523b02 admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:53:00.423357 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca88d96c-6171-4ff1-8c8c-67cf60523b02 admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:53:00.428236 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7101f18e-0d5e-47c3-9f33-1454fb0de8f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:53:00.428862 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7101f18e-0d5e-47c3-9f33-1454fb0de8f9 admin admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:53:00.429145 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7101f18e-0d5e-47c3-9f33-1454fb0de8f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:53:00.429415 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7101f18e-0d5e-47c3-9f33-1454fb0de8f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:53:00.430045 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7101f18e-0d5e-47c3-9f33-1454fb0de8f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:53:00.432143 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7101f18e-0d5e-47c3-9f33-1454fb0de8f9 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:53:00.442419 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1334/2667] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Aug 24 06:53:00 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:53:00.446792 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72310fd9-739e-4bd5-8280-6a4c8c52564d admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:53:00.457585 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72310fd9-739e-4bd5-8280-6a4c8c52564d admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:53:00.462184 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a055ddc0-790e-4ab5-b9b1-efce611814f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:53:00.462836 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a055ddc0-790e-4ab5-b9b1-efce611814f6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:53:00.463111 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a055ddc0-790e-4ab5-b9b1-efce611814f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:53:00.463386 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a055ddc0-790e-4ab5-b9b1-efce611814f6 admin admin] PATH_INFO: `/v3/projects/e849789bcac84e1d841de3adb10881ec/users/d9441e21399641269c7b57bde05fbd53/roles/580aa7dbb38d4cff8f9d1bb8983f99ef` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:53:00.477932 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a055ddc0-790e-4ab5-b9b1-efce611814f6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e849789bcac84e1d841de3adb10881ec, user_id=d9441e21399641269c7b57bde05fbd53, role_id=580aa7dbb38d4cff8f9d1bb8983f99ef)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:53:00.480622 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a055ddc0-790e-4ab5-b9b1-efce611814f6 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:53:00.548188 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1334/2668] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Aug 24 06:53:00 2022] PUT /identity/v3/projects/e849789bcac84e1d841de3adb10881ec/users/d9441e21399641269c7b57bde05fbd53/roles/580aa7dbb38d4cff8f9d1bb8983f99ef => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:53:00.558898 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d836036d-0303-44f9-8229-be4424b01625 None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:53:00.559212 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d836036d-0303-44f9-8229-be4424b01625 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:53:00.559487 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d836036d-0303-44f9-8229-be4424b01625 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:53:00.591426 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-d836036d-0303-44f9-8229-be4424b01625 None None] MFA Rules not processed for user `d9441e21399641269c7b57bde05fbd53`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:53:00.592684 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-d836036d-0303-44f9-8229-be4424b01625 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:53:00.658734 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1335/2669] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:53:00 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:53:00.670310 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a055ddc0-790e-4ab5-b9b1-efce611814f6 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:53:00.672005 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-a055ddc0-790e-4ab5-b9b1-efce611814f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:53:00.734222 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a055ddc0-790e-4ab5-b9b1-efce611814f6 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:53:00.737953 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d65e6f4-4e83-4294-be0b-64df4c3969d5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c4def55ac0e48b29836bc5c64bc7402', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:53:00.738560 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d65e6f4-4e83-4294-be0b-64df4c3969d5 service cinder] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:53:00.738835 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d65e6f4-4e83-4294-be0b-64df4c3969d5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:53:00.739102 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d65e6f4-4e83-4294-be0b-64df4c3969d5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:53:00.748652 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d65e6f4-4e83-4294-be0b-64df4c3969d5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:53:00.750091 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d65e6f4-4e83-4294-be0b-64df4c3969d5 service cinder] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:53:00.759438 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1335/2670] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:53:00 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:53:10.655661 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=66,keystone:INSERT=3 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:53:10.755023 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:53:37.401641 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d836036d-0303-44f9-8229-be4424b01625 None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:53:37.411281 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d836036d-0303-44f9-8229-be4424b01625 None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:53:37.414777 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443d3a68-b0f2-454a-b210-ce6da4718a70 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c4def55ac0e48b29836bc5c64bc7402', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:53:37.415370 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443d3a68-b0f2-454a-b210-ce6da4718a70 service cinder] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:53:37.416629 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443d3a68-b0f2-454a-b210-ce6da4718a70 service cinder] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:53:37.416904 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443d3a68-b0f2-454a-b210-ce6da4718a70 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:53:37.418757 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-443d3a68-b0f2-454a-b210-ce6da4718a70 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:53:37.505825 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443d3a68-b0f2-454a-b210-ce6da4718a70 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:53:37.507293 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443d3a68-b0f2-454a-b210-ce6da4718a70 service cinder] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:53:37.526760 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1336/2671] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:53:37 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:53:40.086098 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d65e6f4-4e83-4294-be0b-64df4c3969d5 service cinder] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:53:40.147284 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d65e6f4-4e83-4294-be0b-64df4c3969d5 service cinder] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:53:40.152010 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a46b3ce-f9d1-41e6-8b53-77e17bf73a8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:53:40.152665 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a46b3ce-f9d1-41e6-8b53-77e17bf73a8f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:53:40.152943 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a46b3ce-f9d1-41e6-8b53-77e17bf73a8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:53:40.153211 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a46b3ce-f9d1-41e6-8b53-77e17bf73a8f admin admin] PATH_INFO: `/v3/users/d9441e21399641269c7b57bde05fbd53` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:53:40.154361 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a46b3ce-f9d1-41e6-8b53-77e17bf73a8f admin admin] RBAC: Authorizing `identity:delete_user(user_id=d9441e21399641269c7b57bde05fbd53)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:53:40.156276 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a46b3ce-f9d1-41e6-8b53-77e17bf73a8f admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:53:40.368181 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-0a46b3ce-f9d1-41e6-8b53-77e17bf73a8f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd9441e21399641269c7b57bde05fbd53'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:53:40.375268 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-0a46b3ce-f9d1-41e6-8b53-77e17bf73a8f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd9441e21399641269c7b57bde05fbd53'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:53:40.375964 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-0a46b3ce-f9d1-41e6-8b53-77e17bf73a8f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd9441e21399641269c7b57bde05fbd53'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:53:40.445072 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-0a46b3ce-f9d1-41e6-8b53-77e17bf73a8f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd9441e21399641269c7b57bde05fbd53'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:53:40.459660 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-0a46b3ce-f9d1-41e6-8b53-77e17bf73a8f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd9441e21399641269c7b57bde05fbd53'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:53:40.466561 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1336/2672] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Aug 24 06:53:40 2022] DELETE /identity/v3/users/d9441e21399641269c7b57bde05fbd53 => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:53:40.471182 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8bae05-53e0-4051-803b-f95cb0e860d4 None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:53:40.471646 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8bae05-53e0-4051-803b-f95cb0e860d4 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:53:40.471761 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8bae05-53e0-4051-803b-f95cb0e860d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:53:40.491070 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-ee8bae05-53e0-4051-803b-f95cb0e860d4 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:53:40.492281 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-ee8bae05-53e0-4051-803b-f95cb0e860d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:53:40.553982 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1337/2673] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:53:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:53:40.563933 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a46b3ce-f9d1-41e6-8b53-77e17bf73a8f admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:53:40.565440 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-0a46b3ce-f9d1-41e6-8b53-77e17bf73a8f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:53:40.623207 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a46b3ce-f9d1-41e6-8b53-77e17bf73a8f admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:53:40.626710 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623f91a6-045c-4082-9437-7af297fccb00 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bb6c79db75e4d169331c02133d1ec32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:53:40.627306 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623f91a6-045c-4082-9437-7af297fccb00 service neutron] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:53:40.627583 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623f91a6-045c-4082-9437-7af297fccb00 service neutron] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:53:40.627854 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623f91a6-045c-4082-9437-7af297fccb00 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:53:40.637416 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623f91a6-045c-4082-9437-7af297fccb00 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:53:40.638890 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623f91a6-045c-4082-9437-7af297fccb00 service neutron] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:53:40.648648 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1337/2674] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:53:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:53:41.098344 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8bae05-53e0-4051-803b-f95cb0e860d4 None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:53:41.099767 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-ee8bae05-53e0-4051-803b-f95cb0e860d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:53:41.115886 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8bae05-53e0-4051-803b-f95cb0e860d4 None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:53:41.122221 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97986d60-c646-442f-9f2d-c7061dbcd618 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:53:41.122950 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97986d60-c646-442f-9f2d-c7061dbcd618 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:53:41.123411 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97986d60-c646-442f-9f2d-c7061dbcd618 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:53:41.123727 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97986d60-c646-442f-9f2d-c7061dbcd618 admin admin] PATH_INFO: `/v3/projects/e849789bcac84e1d841de3adb10881ec` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:53:41.125317 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97986d60-c646-442f-9f2d-c7061dbcd618 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e849789bcac84e1d841de3adb10881ec)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:53:41.127233 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97986d60-c646-442f-9f2d-c7061dbcd618 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:53:41.301298 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-97986d60-c646-442f-9f2d-c7061dbcd618 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e849789bcac84e1d841de3adb10881ec', 'request_id': 'req-97986d60-c646-442f-9f2d-c7061dbcd618'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:53:41.307406 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-97986d60-c646-442f-9f2d-c7061dbcd618 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e849789bcac84e1d841de3adb10881ec', 'request_id': 'req-97986d60-c646-442f-9f2d-c7061dbcd618'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:53:41.313947 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-97986d60-c646-442f-9f2d-c7061dbcd618 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e849789bcac84e1d841de3adb10881ec', 'request_id': 'req-97986d60-c646-442f-9f2d-c7061dbcd618'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:53:41.318286 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:53:41.433830 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-97986d60-c646-442f-9f2d-c7061dbcd618 admin admin] The token cache is being invalidate because project e849789bcac84e1d841de3adb10881ec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81851) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:53:41.434125 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-97986d60-c646-442f-9f2d-c7061dbcd618 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:53:41.434758 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-97986d60-c646-442f-9f2d-c7061dbcd618 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:53:41.436314 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1338/2675] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Aug 24 06:53:41 2022] DELETE /identity/v3/projects/e849789bcac84e1d841de3adb10881ec => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:53:41.441540 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdf30d5-fa93-447f-97d3-13c3fb57466f None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:53:41.442038 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdf30d5-fa93-447f-97d3-13c3fb57466f None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:53:41.442137 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdf30d5-fa93-447f-97d3-13c3fb57466f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:53:41.462014 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-0bdf30d5-fa93-447f-97d3-13c3fb57466f None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:53:41.463272 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-0bdf30d5-fa93-447f-97d3-13c3fb57466f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:53:41.482440 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:53:41.526159 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1338/2676] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:53:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:53:41.533945 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9999b52-4913-49a3-ba95-4a685aa37317 None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:53:41.534430 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9999b52-4913-49a3-ba95-4a685aa37317 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:53:41.534525 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9999b52-4913-49a3-ba95-4a685aa37317 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:53:41.554571 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-a9999b52-4913-49a3-ba95-4a685aa37317 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:53:41.555760 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-a9999b52-4913-49a3-ba95-4a685aa37317 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:53:41.564209 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1339/2677] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:53:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:53:41.568376 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdf30d5-fa93-447f-97d3-13c3fb57466f None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:53:41.577861 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdf30d5-fa93-447f-97d3-13c3fb57466f None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:53:41.582508 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de8e1b4-b76c-4950-b8af-885ac1c5c40f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:53:41.583105 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de8e1b4-b76c-4950-b8af-885ac1c5c40f admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:53:41.583375 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de8e1b4-b76c-4950-b8af-885ac1c5c40f admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:53:41.583653 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de8e1b4-b76c-4950-b8af-885ac1c5c40f admin admin] PATH_INFO: `/v3/domains` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:53:41.584282 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de8e1b4-b76c-4950-b8af-885ac1c5c40f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:53:41.584608 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de8e1b4-b76c-4950-b8af-885ac1c5c40f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:53:41.586192 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de8e1b4-b76c-4950-b8af-885ac1c5c40f admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:53:41.597958 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1339/2678] 10.222.0.62 () {58 vars in 1153 bytes} [Wed Aug 24 06:53:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:53:41.602275 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9999b52-4913-49a3-ba95-4a685aa37317 None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:53:41.612095 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9999b52-4913-49a3-ba95-4a685aa37317 None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:53:41.616903 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6836cd14-94fb-4a11-a231-45ab0ea1426e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:53:41.621573 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6836cd14-94fb-4a11-a231-45ab0ea1426e admin admin] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:53:41.621573 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6836cd14-94fb-4a11-a231-45ab0ea1426e admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:53:41.621573 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6836cd14-94fb-4a11-a231-45ab0ea1426e admin admin] PATH_INFO: `/v3/projects` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:53:41.621573 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6836cd14-94fb-4a11-a231-45ab0ea1426e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:53:41.621573 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6836cd14-94fb-4a11-a231-45ab0ea1426e admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:53:41.733888 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1340/2679] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Aug 24 06:53:41 2022] POST /identity/v3/projects => generated 371 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:53:41.738245 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de8e1b4-b76c-4950-b8af-885ac1c5c40f admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:53:41.750286 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de8e1b4-b76c-4950-b8af-885ac1c5c40f admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:53:41.808907 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1651391e-4994-4507-a1a4-8572065fd718 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:53:41.809558 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1651391e-4994-4507-a1a4-8572065fd718 admin admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:53:41.809828 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1651391e-4994-4507-a1a4-8572065fd718 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:53:41.810096 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1651391e-4994-4507-a1a4-8572065fd718 admin admin] PATH_INFO: `/v3/users` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:53:41.810718 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1651391e-4994-4507-a1a4-8572065fd718 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:53:41.812987 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1651391e-4994-4507-a1a4-8572065fd718 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:53:41.906911 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1340/2680] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Aug 24 06:53:41 2022] POST /identity/v3/users => generated 286 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:53:41.910943 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6836cd14-94fb-4a11-a231-45ab0ea1426e admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:53:41.922112 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6836cd14-94fb-4a11-a231-45ab0ea1426e admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:53:41.926874 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6277f5a5-fcc4-4813-9d53-fc74bce94b35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:53:41.927468 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6277f5a5-fcc4-4813-9d53-fc74bce94b35 admin admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:53:41.927748 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6277f5a5-fcc4-4813-9d53-fc74bce94b35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:53:41.928014 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6277f5a5-fcc4-4813-9d53-fc74bce94b35 admin admin] PATH_INFO: `/v3/roles` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:53:41.928695 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6277f5a5-fcc4-4813-9d53-fc74bce94b35 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:53:41.930288 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6277f5a5-fcc4-4813-9d53-fc74bce94b35 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:53:41.941529 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1341/2681] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Aug 24 06:53:41 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:53:41.945600 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1651391e-4994-4507-a1a4-8572065fd718 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:53:41.956677 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1651391e-4994-4507-a1a4-8572065fd718 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:53:41.961529 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b779ff-eda2-434d-b355-bdcdd9ce8c24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:53:41.962075 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b779ff-eda2-434d-b355-bdcdd9ce8c24 admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:53:41.962362 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b779ff-eda2-434d-b355-bdcdd9ce8c24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:53:41.963242 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b779ff-eda2-434d-b355-bdcdd9ce8c24 admin admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:53:41.963933 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b779ff-eda2-434d-b355-bdcdd9ce8c24 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:53:41.965593 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b779ff-eda2-434d-b355-bdcdd9ce8c24 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:53:41.976802 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1341/2682] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Aug 24 06:53:41 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:53:41.981139 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6277f5a5-fcc4-4813-9d53-fc74bce94b35 admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:53:41.991398 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6277f5a5-fcc4-4813-9d53-fc74bce94b35 admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:53:41.996179 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70950b8a-8b21-401d-8abd-1d49c3eece77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:53:41.996873 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70950b8a-8b21-401d-8abd-1d49c3eece77 admin admin] REQUEST_METHOD: `PUT` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:53:41.997182 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70950b8a-8b21-401d-8abd-1d49c3eece77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:53:41.997459 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70950b8a-8b21-401d-8abd-1d49c3eece77 admin admin] PATH_INFO: `/v3/projects/9f3da901975c416a99fb039d67f6c340/users/16e513b5344a40a4ad0244bb978898bc/roles/580aa7dbb38d4cff8f9d1bb8983f99ef` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:53:42.013184 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70950b8a-8b21-401d-8abd-1d49c3eece77 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f3da901975c416a99fb039d67f6c340, user_id=16e513b5344a40a4ad0244bb978898bc, role_id=580aa7dbb38d4cff8f9d1bb8983f99ef)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:53:42.015861 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70950b8a-8b21-401d-8abd-1d49c3eece77 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:53:42.148817 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1342/2683] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Aug 24 06:53:41 2022] PUT /identity/v3/projects/9f3da901975c416a99fb039d67f6c340/users/16e513b5344a40a4ad0244bb978898bc/roles/580aa7dbb38d4cff8f9d1bb8983f99ef => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:53:42.160023 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc50c43-e19d-4486-b8ff-b3ec78e0fa8c None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:53:42.160310 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc50c43-e19d-4486-b8ff-b3ec78e0fa8c None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:53:42.160618 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc50c43-e19d-4486-b8ff-b3ec78e0fa8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:53:42.189102 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-6cc50c43-e19d-4486-b8ff-b3ec78e0fa8c None None] MFA Rules not processed for user `16e513b5344a40a4ad0244bb978898bc`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:53:42.190317 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-6cc50c43-e19d-4486-b8ff-b3ec78e0fa8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:53:42.253410 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1342/2684] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:53:42 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 06:53:42.265022 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70950b8a-8b21-401d-8abd-1d49c3eece77 admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:53:42.266700 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-70950b8a-8b21-401d-8abd-1d49c3eece77 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:53:42.324688 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70950b8a-8b21-401d-8abd-1d49c3eece77 admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:53:42.328394 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c746f7-1bfb-4625-bb28-633a18a8a062 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c4def55ac0e48b29836bc5c64bc7402', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:53:42.329037 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c746f7-1bfb-4625-bb28-633a18a8a062 service cinder] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:53:42.329316 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c746f7-1bfb-4625-bb28-633a18a8a062 service cinder] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:53:42.329587 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c746f7-1bfb-4625-bb28-633a18a8a062 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:53:42.337848 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c746f7-1bfb-4625-bb28-633a18a8a062 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:53:42.339280 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c746f7-1bfb-4625-bb28-633a18a8a062 service cinder] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:53:42.348234 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1343/2685] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:53:42 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:53:42.373418 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc50c43-e19d-4486-b8ff-b3ec78e0fa8c None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:53:42.392284 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:53:42.440047 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc50c43-e19d-4486-b8ff-b3ec78e0fa8c None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:53:42.446528 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2074c66b-939c-454f-ab97-542b083e1f21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:53:42.447204 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2074c66b-939c-454f-ab97-542b083e1f21 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:53:42.447483 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2074c66b-939c-454f-ab97-542b083e1f21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:53:42.447755 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2074c66b-939c-454f-ab97-542b083e1f21 admin admin] PATH_INFO: `/v3/users/16e513b5344a40a4ad0244bb978898bc` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:53:42.448981 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2074c66b-939c-454f-ab97-542b083e1f21 admin admin] RBAC: Authorizing `identity:delete_user(user_id=16e513b5344a40a4ad0244bb978898bc)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:53:42.450883 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2074c66b-939c-454f-ab97-542b083e1f21 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:53:42.663456 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-2074c66b-939c-454f-ab97-542b083e1f21 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '16e513b5344a40a4ad0244bb978898bc'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:53:42.671081 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-2074c66b-939c-454f-ab97-542b083e1f21 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '16e513b5344a40a4ad0244bb978898bc'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:53:42.671740 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-2074c66b-939c-454f-ab97-542b083e1f21 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '16e513b5344a40a4ad0244bb978898bc'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:53:42.757025 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-2074c66b-939c-454f-ab97-542b083e1f21 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '16e513b5344a40a4ad0244bb978898bc'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:53:42.771468 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-2074c66b-939c-454f-ab97-542b083e1f21 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '16e513b5344a40a4ad0244bb978898bc'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:53:42.778166 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1343/2686] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Aug 24 06:53:42 2022] DELETE /identity/v3/users/16e513b5344a40a4ad0244bb978898bc => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:53:42.782852 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717028b5-307f-4599-978a-8cc1f7c72ccc None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:53:42.783343 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717028b5-307f-4599-978a-8cc1f7c72ccc None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:53:42.783430 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717028b5-307f-4599-978a-8cc1f7c72ccc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:53:42.804755 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-717028b5-307f-4599-978a-8cc1f7c72ccc None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:53:42.805993 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-717028b5-307f-4599-978a-8cc1f7c72ccc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:53:42.868887 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1344/2687] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:53:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:53:42.879101 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2074c66b-939c-454f-ab97-542b083e1f21 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:53:42.880570 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-2074c66b-939c-454f-ab97-542b083e1f21 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:53:42.942817 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2074c66b-939c-454f-ab97-542b083e1f21 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:53:42.947401 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bb98b4-37bd-40c2-a056-e298dffe4a8b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bb6c79db75e4d169331c02133d1ec32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:53:42.948014 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bb98b4-37bd-40c2-a056-e298dffe4a8b service neutron] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:53:42.948479 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bb98b4-37bd-40c2-a056-e298dffe4a8b service neutron] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:53:42.948780 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bb98b4-37bd-40c2-a056-e298dffe4a8b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:53:42.966641 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8bb98b4-37bd-40c2-a056-e298dffe4a8b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:53:42.966641 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8bb98b4-37bd-40c2-a056-e298dffe4a8b service neutron] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:53:42.971607 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1344/2688] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:53:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:53:43.455701 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717028b5-307f-4599-978a-8cc1f7c72ccc None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:53:43.457281 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-717028b5-307f-4599-978a-8cc1f7c72ccc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:53:43.470596 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717028b5-307f-4599-978a-8cc1f7c72ccc None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:53:43.475378 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96010cd1-a2d7-4766-b0f2-256df1c0c887 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:53:43.476124 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96010cd1-a2d7-4766-b0f2-256df1c0c887 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:53:43.477288 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96010cd1-a2d7-4766-b0f2-256df1c0c887 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:53:43.477931 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96010cd1-a2d7-4766-b0f2-256df1c0c887 admin admin] PATH_INFO: `/v3/projects/9f3da901975c416a99fb039d67f6c340` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:53:43.479871 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96010cd1-a2d7-4766-b0f2-256df1c0c887 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9f3da901975c416a99fb039d67f6c340)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:53:43.482276 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96010cd1-a2d7-4766-b0f2-256df1c0c887 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:53:43.571162 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:DELETE=3,keystone:SELECT=95,keystone:INSERT=2 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:53:43.587110 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-96010cd1-a2d7-4766-b0f2-256df1c0c887 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9f3da901975c416a99fb039d67f6c340', 'request_id': 'req-96010cd1-a2d7-4766-b0f2-256df1c0c887'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:53:43.592705 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-96010cd1-a2d7-4766-b0f2-256df1c0c887 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9f3da901975c416a99fb039d67f6c340', 'request_id': 'req-96010cd1-a2d7-4766-b0f2-256df1c0c887'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:53:43.598886 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-96010cd1-a2d7-4766-b0f2-256df1c0c887 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9f3da901975c416a99fb039d67f6c340', 'request_id': 'req-96010cd1-a2d7-4766-b0f2-256df1c0c887'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:53:43.699354 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-96010cd1-a2d7-4766-b0f2-256df1c0c887 admin admin] The token cache is being invalidate because project 9f3da901975c416a99fb039d67f6c340 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81851) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:53:43.699644 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-96010cd1-a2d7-4766-b0f2-256df1c0c887 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:53:43.700268 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-96010cd1-a2d7-4766-b0f2-256df1c0c887 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:53:43.701798 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1345/2689] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Aug 24 06:53:43 2022] DELETE /identity/v3/projects/9f3da901975c416a99fb039d67f6c340 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:53:52.970650 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=71,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:53:53.765411 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=14,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:54:45.589291 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bb98b4-37bd-40c2-a056-e298dffe4a8b service neutron] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:54:45.590704 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-a8bb98b4-37bd-40c2-a056-e298dffe4a8b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:54:45.602352 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bb98b4-37bd-40c2-a056-e298dffe4a8b service neutron] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:54:45.650027 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b3d7fa-c99b-46a7-9f9e-3a42d8cc3b90 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bb6c79db75e4d169331c02133d1ec32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:54:45.650652 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b3d7fa-c99b-46a7-9f9e-3a42d8cc3b90 service neutron] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:54:45.650931 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b3d7fa-c99b-46a7-9f9e-3a42d8cc3b90 service neutron] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:54:45.651202 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b3d7fa-c99b-46a7-9f9e-3a42d8cc3b90 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:54:45.652945 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-05b3d7fa-c99b-46a7-9f9e-3a42d8cc3b90 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:54:45.714401 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b3d7fa-c99b-46a7-9f9e-3a42d8cc3b90 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:54:45.715831 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b3d7fa-c99b-46a7-9f9e-3a42d8cc3b90 service neutron] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:54:45.725374 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1345/2690] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:54:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:54:49.213744 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96010cd1-a2d7-4766-b0f2-256df1c0c887 admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:54:49.215313 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-96010cd1-a2d7-4766-b0f2-256df1c0c887 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:54:49.286199 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96010cd1-a2d7-4766-b0f2-256df1c0c887 admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:54:49.291763 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8263d9-fa0a-4cb8-abc3-6f5332e0b694 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abe9d8c9291845dfa4ef55034e5c9e32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:54:49.292395 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8263d9-fa0a-4cb8-abc3-6f5332e0b694 service placement] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:54:49.292785 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8263d9-fa0a-4cb8-abc3-6f5332e0b694 service placement] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:54:49.293058 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8263d9-fa0a-4cb8-abc3-6f5332e0b694 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:54:49.294877 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-ed8263d9-fa0a-4cb8-abc3-6f5332e0b694 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:54:49.306673 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8263d9-fa0a-4cb8-abc3-6f5332e0b694 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:54:49.308232 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8263d9-fa0a-4cb8-abc3-6f5332e0b694 service placement] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:54:49.318545 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1346/2691] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:54:49 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:54:49.348617 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b3d7fa-c99b-46a7-9f9e-3a42d8cc3b90 service neutron] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:54:49.350490 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-05b3d7fa-c99b-46a7-9f9e-3a42d8cc3b90 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:54:49.364620 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b3d7fa-c99b-46a7-9f9e-3a42d8cc3b90 service neutron] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:54:49.369193 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b088012b-0280-4369-ac88-ba0e93a36001 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:54:49.369818 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b088012b-0280-4369-ac88-ba0e93a36001 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:54:49.370090 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b088012b-0280-4369-ac88-ba0e93a36001 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:54:49.370358 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b088012b-0280-4369-ac88-ba0e93a36001 admin admin] PATH_INFO: `/v3/users/5059723f417445fa934ecfe3e50d0197` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:54:49.371454 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b088012b-0280-4369-ac88-ba0e93a36001 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5059723f417445fa934ecfe3e50d0197)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:54:49.373407 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b088012b-0280-4369-ac88-ba0e93a36001 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:54:49.566833 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-b088012b-0280-4369-ac88-ba0e93a36001 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5059723f417445fa934ecfe3e50d0197'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:54:49.573569 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-b088012b-0280-4369-ac88-ba0e93a36001 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5059723f417445fa934ecfe3e50d0197'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:54:49.574226 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-b088012b-0280-4369-ac88-ba0e93a36001 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5059723f417445fa934ecfe3e50d0197'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:54:49.632157 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-b088012b-0280-4369-ac88-ba0e93a36001 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5059723f417445fa934ecfe3e50d0197'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:54:49.644627 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-b088012b-0280-4369-ac88-ba0e93a36001 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5059723f417445fa934ecfe3e50d0197'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:54:49.650561 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1346/2692] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Aug 24 06:54:49 2022] DELETE /identity/v3/users/5059723f417445fa934ecfe3e50d0197 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:54:49.899262 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8263d9-fa0a-4cb8-abc3-6f5332e0b694 service placement] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:54:49.917730 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8263d9-fa0a-4cb8-abc3-6f5332e0b694 service placement] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:54:49.985994 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631ccd07-4dad-4eb5-a62a-416a9df44efd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:54:49.986655 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631ccd07-4dad-4eb5-a62a-416a9df44efd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:54:49.986932 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631ccd07-4dad-4eb5-a62a-416a9df44efd admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:54:49.987206 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631ccd07-4dad-4eb5-a62a-416a9df44efd admin admin] PATH_INFO: `/v3/projects/f4e8d9878aff47cea1bd895f618badb8` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:54:49.988416 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631ccd07-4dad-4eb5-a62a-416a9df44efd admin admin] RBAC: Authorizing `identity:delete_project(project_id=f4e8d9878aff47cea1bd895f618badb8)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:54:49.990412 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631ccd07-4dad-4eb5-a62a-416a9df44efd admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:54:50.177411 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-631ccd07-4dad-4eb5-a62a-416a9df44efd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f4e8d9878aff47cea1bd895f618badb8', 'request_id': 'req-631ccd07-4dad-4eb5-a62a-416a9df44efd'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:54:50.183657 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-631ccd07-4dad-4eb5-a62a-416a9df44efd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f4e8d9878aff47cea1bd895f618badb8', 'request_id': 'req-631ccd07-4dad-4eb5-a62a-416a9df44efd'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:54:50.190115 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-631ccd07-4dad-4eb5-a62a-416a9df44efd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f4e8d9878aff47cea1bd895f618badb8', 'request_id': 'req-631ccd07-4dad-4eb5-a62a-416a9df44efd'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:54:50.260409 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-631ccd07-4dad-4eb5-a62a-416a9df44efd admin admin] The token cache is being invalidate because project f4e8d9878aff47cea1bd895f618badb8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81851) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:54:50.261039 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-631ccd07-4dad-4eb5-a62a-416a9df44efd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:54:50.261377 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-631ccd07-4dad-4eb5-a62a-416a9df44efd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:54:50.262864 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1347/2693] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Aug 24 06:54:49 2022] DELETE /identity/v3/projects/f4e8d9878aff47cea1bd895f618badb8 => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:54:50.268669 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a244fe-afef-45a0-ab54-f410bd1a8662 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:54:50.271387 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a244fe-afef-45a0-ab54-f410bd1a8662 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:54:50.271387 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a244fe-afef-45a0-ab54-f410bd1a8662 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:54:50.289575 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-50a244fe-afef-45a0-ab54-f410bd1a8662 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:54:50.290782 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-50a244fe-afef-45a0-ab54-f410bd1a8662 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:54:50.307683 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:54:50.349852 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1347/2694] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:54:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 06:54:50.358656 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a937905-7f12-4edc-afd3-389c1ea0d09e None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:54:50.358940 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a937905-7f12-4edc-afd3-389c1ea0d09e None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:54:50.359215 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a937905-7f12-4edc-afd3-389c1ea0d09e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:54:50.379291 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-8a937905-7f12-4edc-afd3-389c1ea0d09e None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:54:50.380512 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-8a937905-7f12-4edc-afd3-389c1ea0d09e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:54:50.388914 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1348/2695] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:54:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:54:50.393168 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a244fe-afef-45a0-ab54-f410bd1a8662 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:54:50.403761 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a244fe-afef-45a0-ab54-f410bd1a8662 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:54:50.408221 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bb4160-d1be-4d0b-a177-64da3b493659 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:54:50.408846 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bb4160-d1be-4d0b-a177-64da3b493659 admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:54:50.409121 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bb4160-d1be-4d0b-a177-64da3b493659 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:54:50.409390 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bb4160-d1be-4d0b-a177-64da3b493659 admin admin] PATH_INFO: `/v3/domains` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:54:50.410027 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4bb4160-d1be-4d0b-a177-64da3b493659 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:54:50.410327 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4bb4160-d1be-4d0b-a177-64da3b493659 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:54:50.411906 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4bb4160-d1be-4d0b-a177-64da3b493659 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:54:50.423627 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1348/2696] 10.222.0.62 () {58 vars in 1153 bytes} [Wed Aug 24 06:54:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 06:54:50.427915 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a937905-7f12-4edc-afd3-389c1ea0d09e None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:54:50.437177 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a937905-7f12-4edc-afd3-389c1ea0d09e None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:54:50.442059 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df5a62a-e155-4740-a4ba-ff6f8a504b26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:54:50.442654 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df5a62a-e155-4740-a4ba-ff6f8a504b26 admin admin] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:54:50.442934 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df5a62a-e155-4740-a4ba-ff6f8a504b26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:54:50.443205 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df5a62a-e155-4740-a4ba-ff6f8a504b26 admin admin] PATH_INFO: `/v3/projects` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:54:50.443831 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df5a62a-e155-4740-a4ba-ff6f8a504b26 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:54:50.445746 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df5a62a-e155-4740-a4ba-ff6f8a504b26 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:54:50.457499 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:54:50.542998 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1349/2697] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Aug 24 06:54:50 2022] POST /identity/v3/projects => generated 383 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:54:50.547342 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bb4160-d1be-4d0b-a177-64da3b493659 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:54:50.557565 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bb4160-d1be-4d0b-a177-64da3b493659 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:54:50.613704 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b3a624-a4aa-47fa-90e9-f0ea716c1b80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:54:50.614233 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b3a624-a4aa-47fa-90e9-f0ea716c1b80 admin admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:54:50.614522 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b3a624-a4aa-47fa-90e9-f0ea716c1b80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:54:50.614809 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b3a624-a4aa-47fa-90e9-f0ea716c1b80 admin admin] PATH_INFO: `/v3/users` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:54:50.615438 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b3a624-a4aa-47fa-90e9-f0ea716c1b80 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:54:50.617418 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b3a624-a4aa-47fa-90e9-f0ea716c1b80 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:54:50.706899 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1349/2698] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Aug 24 06:54:50 2022] POST /identity/v3/users => generated 292 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:54:50.711073 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df5a62a-e155-4740-a4ba-ff6f8a504b26 admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:54:50.722059 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df5a62a-e155-4740-a4ba-ff6f8a504b26 admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:54:50.726718 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229757c5-496c-4d64-9f80-f5b248e4a1e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:54:50.727345 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229757c5-496c-4d64-9f80-f5b248e4a1e6 admin admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:54:50.727621 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229757c5-496c-4d64-9f80-f5b248e4a1e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:54:50.727893 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229757c5-496c-4d64-9f80-f5b248e4a1e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:54:50.729177 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229757c5-496c-4d64-9f80-f5b248e4a1e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:54:50.730813 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229757c5-496c-4d64-9f80-f5b248e4a1e6 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:54:50.741061 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1350/2699] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Aug 24 06:54:50 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:54:50.745363 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b3a624-a4aa-47fa-90e9-f0ea716c1b80 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:54:50.757549 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b3a624-a4aa-47fa-90e9-f0ea716c1b80 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:54:50.762035 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f255c3b-6b53-477c-ae4b-848cbf6f851f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:54:50.762805 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f255c3b-6b53-477c-ae4b-848cbf6f851f admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:54:50.763084 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f255c3b-6b53-477c-ae4b-848cbf6f851f admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:54:50.763357 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f255c3b-6b53-477c-ae4b-848cbf6f851f admin admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:54:50.763987 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f255c3b-6b53-477c-ae4b-848cbf6f851f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:54:50.765636 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f255c3b-6b53-477c-ae4b-848cbf6f851f admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:54:50.776620 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1350/2700] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Aug 24 06:54:50 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:54:50.780798 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229757c5-496c-4d64-9f80-f5b248e4a1e6 admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:54:50.790632 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229757c5-496c-4d64-9f80-f5b248e4a1e6 admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:54:50.795745 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578a5ce6-a32a-4b83-8c06-34509865cfca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:54:50.796389 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578a5ce6-a32a-4b83-8c06-34509865cfca admin admin] REQUEST_METHOD: `PUT` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:54:50.796702 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578a5ce6-a32a-4b83-8c06-34509865cfca admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:54:50.796978 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578a5ce6-a32a-4b83-8c06-34509865cfca admin admin] PATH_INFO: `/v3/projects/b497d4d51a85438a816a14e2455dd948/users/e4837de70e5a460e899d696c7337bf78/roles/580aa7dbb38d4cff8f9d1bb8983f99ef` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:54:50.811519 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578a5ce6-a32a-4b83-8c06-34509865cfca admin admin] RBAC: Authorizing `identity:create_grant(project_id=b497d4d51a85438a816a14e2455dd948, user_id=e4837de70e5a460e899d696c7337bf78, role_id=580aa7dbb38d4cff8f9d1bb8983f99ef)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:54:50.814279 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578a5ce6-a32a-4b83-8c06-34509865cfca admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:54:50.903226 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1351/2701] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Aug 24 06:54:50 2022] PUT /identity/v3/projects/b497d4d51a85438a816a14e2455dd948/users/e4837de70e5a460e899d696c7337bf78/roles/580aa7dbb38d4cff8f9d1bb8983f99ef => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:54:50.913187 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15c75cd-80f0-4be9-b8ef-0f9795893a5b None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:54:50.913477 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15c75cd-80f0-4be9-b8ef-0f9795893a5b None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:54:50.913746 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15c75cd-80f0-4be9-b8ef-0f9795893a5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:54:50.942063 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-c15c75cd-80f0-4be9-b8ef-0f9795893a5b None None] MFA Rules not processed for user `e4837de70e5a460e899d696c7337bf78`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:54:50.943288 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-c15c75cd-80f0-4be9-b8ef-0f9795893a5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:54:51.008480 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1351/2702] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:54:50 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:54:51.019888 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578a5ce6-a32a-4b83-8c06-34509865cfca admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:54:51.021889 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-578a5ce6-a32a-4b83-8c06-34509865cfca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:54:51.096452 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578a5ce6-a32a-4b83-8c06-34509865cfca admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:54:51.102296 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e391bea-e700-44b2-a0c6-112f6055e99b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb6d3743340241f98cacb9e0d063298c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:54:51.102900 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e391bea-e700-44b2-a0c6-112f6055e99b service nova] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:54:51.103265 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e391bea-e700-44b2-a0c6-112f6055e99b service nova] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:54:51.103542 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e391bea-e700-44b2-a0c6-112f6055e99b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:54:51.111284 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e391bea-e700-44b2-a0c6-112f6055e99b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:54:51.112893 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e391bea-e700-44b2-a0c6-112f6055e99b service nova] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:54:51.121672 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1352/2703] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:54:51 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:54:51.142614 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15c75cd-80f0-4be9-b8ef-0f9795893a5b None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:54:51.144042 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-c15c75cd-80f0-4be9-b8ef-0f9795893a5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:54:51.164316 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:54:51.206472 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15c75cd-80f0-4be9-b8ef-0f9795893a5b None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:54:51.210249 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b37f9b7-179b-483c-9666-c24f4f231219 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c4def55ac0e48b29836bc5c64bc7402', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:54:51.210986 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b37f9b7-179b-483c-9666-c24f4f231219 service cinder] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:54:51.211273 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b37f9b7-179b-483c-9666-c24f4f231219 service cinder] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:54:51.211546 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b37f9b7-179b-483c-9666-c24f4f231219 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:54:51.213959 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-3b37f9b7-179b-483c-9666-c24f4f231219 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:54:51.228128 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b37f9b7-179b-483c-9666-c24f4f231219 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:54:51.229605 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b37f9b7-179b-483c-9666-c24f4f231219 service cinder] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:54:51.241931 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1352/2704] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:54:51 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:54:51.915578 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e391bea-e700-44b2-a0c6-112f6055e99b service nova] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:54:51.925684 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e391bea-e700-44b2-a0c6-112f6055e99b service nova] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:54:51.929275 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d2b100-09a8-4bfe-b828-daaeb73dc401 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c4def55ac0e48b29836bc5c64bc7402', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:54:51.929932 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d2b100-09a8-4bfe-b828-daaeb73dc401 service cinder] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:54:51.930129 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d2b100-09a8-4bfe-b828-daaeb73dc401 service cinder] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:54:51.930397 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d2b100-09a8-4bfe-b828-daaeb73dc401 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:54:51.932725 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-82d2b100-09a8-4bfe-b828-daaeb73dc401 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:54:51.947554 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d2b100-09a8-4bfe-b828-daaeb73dc401 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:54:51.949025 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d2b100-09a8-4bfe-b828-daaeb73dc401 service cinder] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:54:51.960349 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1353/2705] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:54:51 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:54:59.322654 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b37f9b7-179b-483c-9666-c24f4f231219 service cinder] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:54:59.388211 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b37f9b7-179b-483c-9666-c24f4f231219 service cinder] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:54:59.392851 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34c208a-73f1-4c8b-9465-431b17169b4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:54:59.393484 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34c208a-73f1-4c8b-9465-431b17169b4c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:54:59.393765 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34c208a-73f1-4c8b-9465-431b17169b4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:54:59.394042 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34c208a-73f1-4c8b-9465-431b17169b4c admin admin] PATH_INFO: `/v3/users/e4837de70e5a460e899d696c7337bf78` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:54:59.395169 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34c208a-73f1-4c8b-9465-431b17169b4c admin admin] RBAC: Authorizing `identity:delete_user(user_id=e4837de70e5a460e899d696c7337bf78)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:54:59.397139 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34c208a-73f1-4c8b-9465-431b17169b4c admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:54:59.543404 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-b34c208a-73f1-4c8b-9465-431b17169b4c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e4837de70e5a460e899d696c7337bf78'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:54:59.551152 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-b34c208a-73f1-4c8b-9465-431b17169b4c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e4837de70e5a460e899d696c7337bf78'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:54:59.551899 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-b34c208a-73f1-4c8b-9465-431b17169b4c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e4837de70e5a460e899d696c7337bf78'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:54:59.635194 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-b34c208a-73f1-4c8b-9465-431b17169b4c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e4837de70e5a460e899d696c7337bf78'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:54:59.649045 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-b34c208a-73f1-4c8b-9465-431b17169b4c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e4837de70e5a460e899d696c7337bf78'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:54:59.655097 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1353/2706] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Aug 24 06:54:59 2022] DELETE /identity/v3/users/e4837de70e5a460e899d696c7337bf78 => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:54:59.660328 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703599a4-99f0-4d3a-bc12-ee048e159467 None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:54:59.660870 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703599a4-99f0-4d3a-bc12-ee048e159467 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:54:59.660954 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703599a4-99f0-4d3a-bc12-ee048e159467 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:54:59.681998 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-703599a4-99f0-4d3a-bc12-ee048e159467 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:54:59.683555 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-703599a4-99f0-4d3a-bc12-ee048e159467 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:54:59.745313 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1354/2707] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:54:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 06:54:59.756624 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34c208a-73f1-4c8b-9465-431b17169b4c admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:54:59.758563 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-b34c208a-73f1-4c8b-9465-431b17169b4c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:54:59.805290 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:54:59.823751 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34c208a-73f1-4c8b-9465-431b17169b4c admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:54:59.827547 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b57b0df-8e10-4c74-bd0d-b42b7b824776 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bb6c79db75e4d169331c02133d1ec32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:54:59.828158 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b57b0df-8e10-4c74-bd0d-b42b7b824776 service neutron] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:54:59.828436 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b57b0df-8e10-4c74-bd0d-b42b7b824776 service neutron] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:54:59.828740 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b57b0df-8e10-4c74-bd0d-b42b7b824776 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:54:59.838554 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b57b0df-8e10-4c74-bd0d-b42b7b824776 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:54:59.841118 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b57b0df-8e10-4c74-bd0d-b42b7b824776 service neutron] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:54:59.855996 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1354/2708] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:54:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 06:55:00.316878 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703599a4-99f0-4d3a-bc12-ee048e159467 None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:55:00.318414 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-703599a4-99f0-4d3a-bc12-ee048e159467 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:55:00.333526 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703599a4-99f0-4d3a-bc12-ee048e159467 None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:55:00.338768 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a0504f-f005-48d3-8c20-9fa890ca0f7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:55:00.339443 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a0504f-f005-48d3-8c20-9fa890ca0f7e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:55:00.339724 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a0504f-f005-48d3-8c20-9fa890ca0f7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:55:00.340000 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a0504f-f005-48d3-8c20-9fa890ca0f7e admin admin] PATH_INFO: `/v3/projects/b497d4d51a85438a816a14e2455dd948` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:55:00.341141 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a0504f-f005-48d3-8c20-9fa890ca0f7e admin admin] RBAC: Authorizing `identity:delete_project(project_id=b497d4d51a85438a816a14e2455dd948)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:55:00.343117 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a0504f-f005-48d3-8c20-9fa890ca0f7e admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:55:00.368584 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=97,keystone:INSERT=2,keystone:DELETE=1 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:55:00.490876 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-38a0504f-f005-48d3-8c20-9fa890ca0f7e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b497d4d51a85438a816a14e2455dd948', 'request_id': 'req-38a0504f-f005-48d3-8c20-9fa890ca0f7e'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:55:00.496449 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-38a0504f-f005-48d3-8c20-9fa890ca0f7e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b497d4d51a85438a816a14e2455dd948', 'request_id': 'req-38a0504f-f005-48d3-8c20-9fa890ca0f7e'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:55:00.503142 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-38a0504f-f005-48d3-8c20-9fa890ca0f7e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b497d4d51a85438a816a14e2455dd948', 'request_id': 'req-38a0504f-f005-48d3-8c20-9fa890ca0f7e'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:55:00.588630 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-38a0504f-f005-48d3-8c20-9fa890ca0f7e admin admin] The token cache is being invalidate because project b497d4d51a85438a816a14e2455dd948 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81851) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:55:00.588844 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-38a0504f-f005-48d3-8c20-9fa890ca0f7e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:55:00.589444 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-38a0504f-f005-48d3-8c20-9fa890ca0f7e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:55:00.590969 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1355/2709] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Aug 24 06:55:00 2022] DELETE /identity/v3/projects/b497d4d51a85438a816a14e2455dd948 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:55:00.596011 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd530c45-3561-4c36-a02e-8664343a6cf6 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:55:00.596518 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd530c45-3561-4c36-a02e-8664343a6cf6 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:55:00.596629 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd530c45-3561-4c36-a02e-8664343a6cf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:55:00.616866 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-cd530c45-3561-4c36-a02e-8664343a6cf6 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:55:00.618071 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-cd530c45-3561-4c36-a02e-8664343a6cf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:55:00.675768 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1355/2710] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:55:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:55:00.683469 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebda793-7c52-4833-bea3-734c327e95ca None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:55:00.683994 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebda793-7c52-4833-bea3-734c327e95ca None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:55:00.684102 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebda793-7c52-4833-bea3-734c327e95ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:55:00.712402 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-bebda793-7c52-4833-bea3-734c327e95ca None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:55:00.715248 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-bebda793-7c52-4833-bea3-734c327e95ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:55:00.723530 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1356/2711] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:55:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:55:00.727872 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd530c45-3561-4c36-a02e-8664343a6cf6 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:55:00.737676 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd530c45-3561-4c36-a02e-8664343a6cf6 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:55:00.742467 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32be8da8-49cf-4683-9a1a-c82277c3c926 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:55:00.743073 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32be8da8-49cf-4683-9a1a-c82277c3c926 admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:55:00.743354 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32be8da8-49cf-4683-9a1a-c82277c3c926 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:55:00.743643 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32be8da8-49cf-4683-9a1a-c82277c3c926 admin admin] PATH_INFO: `/v3/domains` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:55:00.744270 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32be8da8-49cf-4683-9a1a-c82277c3c926 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:55:00.744608 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32be8da8-49cf-4683-9a1a-c82277c3c926 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:55:00.746201 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32be8da8-49cf-4683-9a1a-c82277c3c926 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:55:00.758183 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1356/2712] 10.222.0.62 () {58 vars in 1153 bytes} [Wed Aug 24 06:55:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:55:00.763282 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebda793-7c52-4833-bea3-734c327e95ca None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:55:00.772766 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebda793-7c52-4833-bea3-734c327e95ca None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:55:00.778011 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a752167-a851-43a3-a991-b4431fd5bf1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:55:00.778603 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a752167-a851-43a3-a991-b4431fd5bf1d admin admin] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:55:00.778876 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a752167-a851-43a3-a991-b4431fd5bf1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:55:00.779142 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a752167-a851-43a3-a991-b4431fd5bf1d admin admin] PATH_INFO: `/v3/projects` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:55:00.779753 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a752167-a851-43a3-a991-b4431fd5bf1d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:55:00.781667 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a752167-a851-43a3-a991-b4431fd5bf1d admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:55:00.893737 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1357/2713] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Aug 24 06:55:00 2022] POST /identity/v3/projects => generated 365 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:55:00.898205 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32be8da8-49cf-4683-9a1a-c82277c3c926 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:55:00.908823 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32be8da8-49cf-4683-9a1a-c82277c3c926 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:55:00.965344 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7651c3f-00f3-48ac-ae3d-647049a45a75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:55:00.965942 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7651c3f-00f3-48ac-ae3d-647049a45a75 admin admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:55:00.966219 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7651c3f-00f3-48ac-ae3d-647049a45a75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:55:00.966489 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7651c3f-00f3-48ac-ae3d-647049a45a75 admin admin] PATH_INFO: `/v3/users` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:55:00.967116 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7651c3f-00f3-48ac-ae3d-647049a45a75 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:55:00.969073 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7651c3f-00f3-48ac-ae3d-647049a45a75 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:55:01.069715 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1357/2714] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Aug 24 06:55:00 2022] POST /identity/v3/users => generated 283 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:55:01.076336 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a752167-a851-43a3-a991-b4431fd5bf1d admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:55:01.086962 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a752167-a851-43a3-a991-b4431fd5bf1d admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:55:01.091528 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b79854-1e36-4343-b804-daca8f502550 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:55:01.092120 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b79854-1e36-4343-b804-daca8f502550 admin admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:55:01.092395 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b79854-1e36-4343-b804-daca8f502550 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:55:01.092701 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b79854-1e36-4343-b804-daca8f502550 admin admin] PATH_INFO: `/v3/roles` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:55:01.093335 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b79854-1e36-4343-b804-daca8f502550 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:55:01.094956 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b79854-1e36-4343-b804-daca8f502550 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:55:01.105276 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1358/2715] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Aug 24 06:55:01 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:55:01.109301 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7651c3f-00f3-48ac-ae3d-647049a45a75 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:55:01.119734 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7651c3f-00f3-48ac-ae3d-647049a45a75 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:55:01.124325 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99aaea7a-0e5c-409e-a816-9798332086bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:55:01.124947 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99aaea7a-0e5c-409e-a816-9798332086bd admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:55:01.125226 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99aaea7a-0e5c-409e-a816-9798332086bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:55:01.125495 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99aaea7a-0e5c-409e-a816-9798332086bd admin admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:55:01.126122 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99aaea7a-0e5c-409e-a816-9798332086bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:55:01.127707 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99aaea7a-0e5c-409e-a816-9798332086bd admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:55:01.138793 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1358/2716] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Aug 24 06:55:01 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 06:55:01.142845 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b79854-1e36-4343-b804-daca8f502550 admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:55:01.152959 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b79854-1e36-4343-b804-daca8f502550 admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:55:01.157419 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7ce1db-c6d1-4da4-b8e4-a1ad196cbdce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:55:01.158057 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7ce1db-c6d1-4da4-b8e4-a1ad196cbdce admin admin] REQUEST_METHOD: `PUT` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:55:01.158333 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7ce1db-c6d1-4da4-b8e4-a1ad196cbdce admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:55:01.158609 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7ce1db-c6d1-4da4-b8e4-a1ad196cbdce admin admin] PATH_INFO: `/v3/projects/ad209b996b1a48978e66dd91ce5a6fd5/users/05da6678de62469c84b1bb07d8adf97f/roles/580aa7dbb38d4cff8f9d1bb8983f99ef` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:55:01.172736 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7ce1db-c6d1-4da4-b8e4-a1ad196cbdce admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad209b996b1a48978e66dd91ce5a6fd5, user_id=05da6678de62469c84b1bb07d8adf97f, role_id=580aa7dbb38d4cff8f9d1bb8983f99ef)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:55:01.175366 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7ce1db-c6d1-4da4-b8e4-a1ad196cbdce admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:55:01.303918 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1359/2717] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Aug 24 06:55:01 2022] PUT /identity/v3/projects/ad209b996b1a48978e66dd91ce5a6fd5/users/05da6678de62469c84b1bb07d8adf97f/roles/580aa7dbb38d4cff8f9d1bb8983f99ef => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:55:01.321111 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47432ad9-5eac-4eaa-896b-bd0401aef081 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:55:01.321393 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47432ad9-5eac-4eaa-896b-bd0401aef081 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:55:01.321667 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47432ad9-5eac-4eaa-896b-bd0401aef081 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:55:01.352157 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-47432ad9-5eac-4eaa-896b-bd0401aef081 None None] MFA Rules not processed for user `05da6678de62469c84b1bb07d8adf97f`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:55:01.354216 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-47432ad9-5eac-4eaa-896b-bd0401aef081 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:55:01.394606 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:55:01.425416 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1359/2718] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:55:01 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:55:01.440986 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7ce1db-c6d1-4da4-b8e4-a1ad196cbdce admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:55:01.447483 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-2e7ce1db-c6d1-4da4-b8e4-a1ad196cbdce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:55:01.508850 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7ce1db-c6d1-4da4-b8e4-a1ad196cbdce admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:55:01.512772 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380837e4-fda7-441f-aed3-5b4827bdd297 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c4def55ac0e48b29836bc5c64bc7402', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:55:01.513381 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380837e4-fda7-441f-aed3-5b4827bdd297 service cinder] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:55:01.513663 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380837e4-fda7-441f-aed3-5b4827bdd297 service cinder] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:55:01.513926 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380837e4-fda7-441f-aed3-5b4827bdd297 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:55:01.522998 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380837e4-fda7-441f-aed3-5b4827bdd297 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:55:01.524434 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380837e4-fda7-441f-aed3-5b4827bdd297 service cinder] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:55:01.534578 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1360/2719] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:55:01 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 06:55:11.378869 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47432ad9-5eac-4eaa-896b-bd0401aef081 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:55:11.440783 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47432ad9-5eac-4eaa-896b-bd0401aef081 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:55:11.445555 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a94116e-31c0-4f50-b212-79b7d952cd1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:55:11.446181 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a94116e-31c0-4f50-b212-79b7d952cd1d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:55:11.446486 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a94116e-31c0-4f50-b212-79b7d952cd1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:55:11.446803 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a94116e-31c0-4f50-b212-79b7d952cd1d admin admin] PATH_INFO: `/v3/users/05da6678de62469c84b1bb07d8adf97f` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:55:11.447897 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a94116e-31c0-4f50-b212-79b7d952cd1d admin admin] RBAC: Authorizing `identity:delete_user(user_id=05da6678de62469c84b1bb07d8adf97f)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:55:11.449831 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a94116e-31c0-4f50-b212-79b7d952cd1d admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:55:11.529395 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=68,keystone:DELETE=6,keystone:INSERT=3 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:55:11.628470 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-4a94116e-31c0-4f50-b212-79b7d952cd1d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '05da6678de62469c84b1bb07d8adf97f'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:55:11.635317 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-4a94116e-31c0-4f50-b212-79b7d952cd1d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '05da6678de62469c84b1bb07d8adf97f'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:55:11.635977 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-4a94116e-31c0-4f50-b212-79b7d952cd1d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '05da6678de62469c84b1bb07d8adf97f'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:55:11.726171 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-4a94116e-31c0-4f50-b212-79b7d952cd1d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '05da6678de62469c84b1bb07d8adf97f'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:55:11.740165 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-4a94116e-31c0-4f50-b212-79b7d952cd1d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '05da6678de62469c84b1bb07d8adf97f'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:55:11.746231 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1360/2720] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Aug 24 06:55:11 2022] DELETE /identity/v3/users/05da6678de62469c84b1bb07d8adf97f => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:55:11.750811 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135277ea-f04d-4bcd-8724-d1f13efbbd1f None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:55:11.751092 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135277ea-f04d-4bcd-8724-d1f13efbbd1f None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:55:11.751367 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135277ea-f04d-4bcd-8724-d1f13efbbd1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:55:11.770257 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-135277ea-f04d-4bcd-8724-d1f13efbbd1f None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:55:11.775596 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-135277ea-f04d-4bcd-8724-d1f13efbbd1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:55:11.829593 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1361/2721] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:55:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:55:11.840677 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a94116e-31c0-4f50-b212-79b7d952cd1d admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:55:11.842053 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-4a94116e-31c0-4f50-b212-79b7d952cd1d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:55:11.898067 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a94116e-31c0-4f50-b212-79b7d952cd1d admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:55:11.902550 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1b7729-186e-4915-a452-8bf0dadfa335 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bb6c79db75e4d169331c02133d1ec32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:55:11.903148 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1b7729-186e-4915-a452-8bf0dadfa335 service neutron] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:55:11.903423 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1b7729-186e-4915-a452-8bf0dadfa335 service neutron] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:55:11.903691 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1b7729-186e-4915-a452-8bf0dadfa335 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:55:11.912971 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1b7729-186e-4915-a452-8bf0dadfa335 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:55:11.914389 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1b7729-186e-4915-a452-8bf0dadfa335 service neutron] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:55:11.919231 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:55:11.923786 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1361/2722] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:55:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:55:12.358697 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135277ea-f04d-4bcd-8724-d1f13efbbd1f None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:55:12.360430 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-135277ea-f04d-4bcd-8724-d1f13efbbd1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:55:12.376245 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135277ea-f04d-4bcd-8724-d1f13efbbd1f None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:55:12.382631 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d21eca-bd5d-48dc-b943-5049cdbba1e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:55:12.383331 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d21eca-bd5d-48dc-b943-5049cdbba1e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:55:12.383609 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d21eca-bd5d-48dc-b943-5049cdbba1e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:55:12.383885 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d21eca-bd5d-48dc-b943-5049cdbba1e4 admin admin] PATH_INFO: `/v3/projects/ad209b996b1a48978e66dd91ce5a6fd5` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:55:12.386356 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d21eca-bd5d-48dc-b943-5049cdbba1e4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ad209b996b1a48978e66dd91ce5a6fd5)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:55:12.388690 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d21eca-bd5d-48dc-b943-5049cdbba1e4 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:55:12.496406 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-a7d21eca-bd5d-48dc-b943-5049cdbba1e4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ad209b996b1a48978e66dd91ce5a6fd5', 'request_id': 'req-a7d21eca-bd5d-48dc-b943-5049cdbba1e4'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:55:12.501822 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-a7d21eca-bd5d-48dc-b943-5049cdbba1e4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ad209b996b1a48978e66dd91ce5a6fd5', 'request_id': 'req-a7d21eca-bd5d-48dc-b943-5049cdbba1e4'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:55:12.508594 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-a7d21eca-bd5d-48dc-b943-5049cdbba1e4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ad209b996b1a48978e66dd91ce5a6fd5', 'request_id': 'req-a7d21eca-bd5d-48dc-b943-5049cdbba1e4'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:55:12.609852 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-a7d21eca-bd5d-48dc-b943-5049cdbba1e4 admin admin] The token cache is being invalidate because project ad209b996b1a48978e66dd91ce5a6fd5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81851) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:55:12.610515 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-a7d21eca-bd5d-48dc-b943-5049cdbba1e4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:55:12.610655 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-a7d21eca-bd5d-48dc-b943-5049cdbba1e4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:55:12.612206 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1362/2723] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Aug 24 06:55:12 2022] DELETE /identity/v3/projects/ad209b996b1a48978e66dd91ce5a6fd5 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:55:12.617162 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0140beeb-e2a3-4004-9d54-f355b6509b41 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:55:12.617591 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0140beeb-e2a3-4004-9d54-f355b6509b41 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:55:12.617711 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0140beeb-e2a3-4004-9d54-f355b6509b41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:55:12.638178 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-0140beeb-e2a3-4004-9d54-f355b6509b41 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:55:12.639364 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-0140beeb-e2a3-4004-9d54-f355b6509b41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:55:12.700336 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1362/2724] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:55:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 06:55:12.708884 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0760544-8673-4216-8158-7eab8aa243b0 None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:55:12.708884 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0760544-8673-4216-8158-7eab8aa243b0 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:55:12.708884 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0760544-8673-4216-8158-7eab8aa243b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:55:12.729004 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-a0760544-8673-4216-8158-7eab8aa243b0 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:55:12.730223 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-a0760544-8673-4216-8158-7eab8aa243b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:55:12.738164 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1363/2725] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:55:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:55:12.742608 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0140beeb-e2a3-4004-9d54-f355b6509b41 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:55:12.753395 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0140beeb-e2a3-4004-9d54-f355b6509b41 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:55:12.757999 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3809b9-9e30-4f1c-97ac-27dbb3d0b917 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:55:12.758600 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3809b9-9e30-4f1c-97ac-27dbb3d0b917 admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:55:12.758879 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3809b9-9e30-4f1c-97ac-27dbb3d0b917 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:55:12.759150 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3809b9-9e30-4f1c-97ac-27dbb3d0b917 admin admin] PATH_INFO: `/v3/domains` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:55:12.759795 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3809b9-9e30-4f1c-97ac-27dbb3d0b917 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:55:12.760095 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3809b9-9e30-4f1c-97ac-27dbb3d0b917 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:55:12.761735 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3809b9-9e30-4f1c-97ac-27dbb3d0b917 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:55:12.775601 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1363/2726] 10.222.0.62 () {58 vars in 1153 bytes} [Wed Aug 24 06:55:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:55:12.783116 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0760544-8673-4216-8158-7eab8aa243b0 None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:55:12.793040 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0760544-8673-4216-8158-7eab8aa243b0 None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:55:12.797906 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0190e991-4d22-46bd-bb20-2cda86c0f67b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:55:12.798507 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0190e991-4d22-46bd-bb20-2cda86c0f67b admin admin] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:55:12.798816 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0190e991-4d22-46bd-bb20-2cda86c0f67b admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:55:12.799086 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0190e991-4d22-46bd-bb20-2cda86c0f67b admin admin] PATH_INFO: `/v3/projects` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:55:12.799705 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0190e991-4d22-46bd-bb20-2cda86c0f67b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:55:12.801715 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0190e991-4d22-46bd-bb20-2cda86c0f67b admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:55:12.885689 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1364/2727] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Aug 24 06:55:12 2022] POST /identity/v3/projects => generated 393 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:55:12.889965 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3809b9-9e30-4f1c-97ac-27dbb3d0b917 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:55:12.900016 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3809b9-9e30-4f1c-97ac-27dbb3d0b917 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:55:12.954845 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2710552-292c-4e41-87f7-2bdd6a0804cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:55:12.955417 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2710552-292c-4e41-87f7-2bdd6a0804cb admin admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:55:12.955693 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2710552-292c-4e41-87f7-2bdd6a0804cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:55:12.955962 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2710552-292c-4e41-87f7-2bdd6a0804cb admin admin] PATH_INFO: `/v3/users` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:55:12.956603 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2710552-292c-4e41-87f7-2bdd6a0804cb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:55:12.958517 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2710552-292c-4e41-87f7-2bdd6a0804cb admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:55:13.036314 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1364/2728] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Aug 24 06:55:12 2022] POST /identity/v3/users => generated 297 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:55:13.037553 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0190e991-4d22-46bd-bb20-2cda86c0f67b admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:55:13.048573 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0190e991-4d22-46bd-bb20-2cda86c0f67b admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:55:13.053209 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b718d6-fe70-435d-a37b-4f1a1f9ded92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:55:13.053964 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b718d6-fe70-435d-a37b-4f1a1f9ded92 admin admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:55:13.054239 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b718d6-fe70-435d-a37b-4f1a1f9ded92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:55:13.054511 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b718d6-fe70-435d-a37b-4f1a1f9ded92 admin admin] PATH_INFO: `/v3/roles` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:55:13.055146 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b718d6-fe70-435d-a37b-4f1a1f9ded92 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:55:13.056761 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b718d6-fe70-435d-a37b-4f1a1f9ded92 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:55:13.068791 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1365/2729] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Aug 24 06:55:13 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:55:13.073760 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2710552-292c-4e41-87f7-2bdd6a0804cb admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:55:13.083779 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2710552-292c-4e41-87f7-2bdd6a0804cb admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:55:13.088974 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdd39cd-c435-458e-94b5-4f6393997fc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:55:13.089564 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdd39cd-c435-458e-94b5-4f6393997fc4 admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:55:13.089844 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdd39cd-c435-458e-94b5-4f6393997fc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:55:13.090115 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdd39cd-c435-458e-94b5-4f6393997fc4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:55:13.090750 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdd39cd-c435-458e-94b5-4f6393997fc4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:55:13.092374 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdd39cd-c435-458e-94b5-4f6393997fc4 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:55:13.104057 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1365/2730] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Aug 24 06:55:13 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:55:13.108111 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b718d6-fe70-435d-a37b-4f1a1f9ded92 admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:55:13.118211 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b718d6-fe70-435d-a37b-4f1a1f9ded92 admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:55:13.122755 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1651c6fd-aaf2-467a-8ff5-cebcdacfafed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:55:13.123389 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1651c6fd-aaf2-467a-8ff5-cebcdacfafed admin admin] REQUEST_METHOD: `PUT` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:55:13.123668 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1651c6fd-aaf2-467a-8ff5-cebcdacfafed admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:55:13.123941 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1651c6fd-aaf2-467a-8ff5-cebcdacfafed admin admin] PATH_INFO: `/v3/projects/530037605bf74e36a19e7459aa5c03c9/users/11ebb079e2474f0c908069edab2f6a68/roles/580aa7dbb38d4cff8f9d1bb8983f99ef` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:55:13.138653 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1651c6fd-aaf2-467a-8ff5-cebcdacfafed admin admin] RBAC: Authorizing `identity:create_grant(project_id=530037605bf74e36a19e7459aa5c03c9, user_id=11ebb079e2474f0c908069edab2f6a68, role_id=580aa7dbb38d4cff8f9d1bb8983f99ef)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:55:13.141418 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1651c6fd-aaf2-467a-8ff5-cebcdacfafed admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:55:13.233230 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1366/2731] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Aug 24 06:55:13 2022] PUT /identity/v3/projects/530037605bf74e36a19e7459aa5c03c9/users/11ebb079e2474f0c908069edab2f6a68/roles/580aa7dbb38d4cff8f9d1bb8983f99ef => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:55:13.244824 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e304b87-0378-4c7c-bf35-9bcf08e39363 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:55:13.245107 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e304b87-0378-4c7c-bf35-9bcf08e39363 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:55:13.245381 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e304b87-0378-4c7c-bf35-9bcf08e39363 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:55:13.272598 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-7e304b87-0378-4c7c-bf35-9bcf08e39363 None None] MFA Rules not processed for user `11ebb079e2474f0c908069edab2f6a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:55:13.273832 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-7e304b87-0378-4c7c-bf35-9bcf08e39363 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:55:13.337127 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:55:13.346836 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1366/2732] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:55:13 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:55:13.351285 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1651c6fd-aaf2-467a-8ff5-cebcdacfafed admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:55:13.373751 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:55:13.414670 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1651c6fd-aaf2-467a-8ff5-cebcdacfafed admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:55:13.419880 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b49b78-4e32-408f-8ac8-d62f91f40061 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:55:13.420481 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b49b78-4e32-408f-8ac8-d62f91f40061 admin admin] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:55:13.420792 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b49b78-4e32-408f-8ac8-d62f91f40061 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:55:13.426024 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b49b78-4e32-408f-8ac8-d62f91f40061 admin admin] PATH_INFO: `/v3/projects` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:55:13.426024 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b49b78-4e32-408f-8ac8-d62f91f40061 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:55:13.426024 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b49b78-4e32-408f-8ac8-d62f91f40061 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:55:13.502174 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1367/2733] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Aug 24 06:55:13 2022] POST /identity/v3/projects => generated 391 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:55:13.506146 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e304b87-0378-4c7c-bf35-9bcf08e39363 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:55:13.516627 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e304b87-0378-4c7c-bf35-9bcf08e39363 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:55:13.572705 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67816d43-2f1d-438a-bc29-2c7bb9ce6c4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:55:13.573306 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67816d43-2f1d-438a-bc29-2c7bb9ce6c4f admin admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:55:13.573583 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67816d43-2f1d-438a-bc29-2c7bb9ce6c4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:55:13.573850 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67816d43-2f1d-438a-bc29-2c7bb9ce6c4f admin admin] PATH_INFO: `/v3/users` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:55:13.574474 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67816d43-2f1d-438a-bc29-2c7bb9ce6c4f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:55:13.576488 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67816d43-2f1d-438a-bc29-2c7bb9ce6c4f admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:55:13.646175 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1367/2734] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Aug 24 06:55:13 2022] POST /identity/v3/users => generated 302 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 06:55:13.650569 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b49b78-4e32-408f-8ac8-d62f91f40061 admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:55:13.661195 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b49b78-4e32-408f-8ac8-d62f91f40061 admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:55:13.668975 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd226c6-1934-4b8a-93df-389cdbb4a95f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:55:13.668975 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd226c6-1934-4b8a-93df-389cdbb4a95f admin admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:55:13.668975 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd226c6-1934-4b8a-93df-389cdbb4a95f admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:55:13.668975 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd226c6-1934-4b8a-93df-389cdbb4a95f admin admin] PATH_INFO: `/v3/roles` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:55:13.669336 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd226c6-1934-4b8a-93df-389cdbb4a95f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:55:13.669789 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd226c6-1934-4b8a-93df-389cdbb4a95f admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:55:13.680461 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1368/2735] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Aug 24 06:55:13 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:55:13.685586 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67816d43-2f1d-438a-bc29-2c7bb9ce6c4f admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:55:13.696181 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67816d43-2f1d-438a-bc29-2c7bb9ce6c4f admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:55:13.700763 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a3d854-8504-4691-99b0-ecf45c698bba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:55:13.701422 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a3d854-8504-4691-99b0-ecf45c698bba admin admin] REQUEST_METHOD: `PUT` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:55:13.701702 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a3d854-8504-4691-99b0-ecf45c698bba admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:55:13.701973 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a3d854-8504-4691-99b0-ecf45c698bba admin admin] PATH_INFO: `/v3/projects/4b4132e03070403db1142a2454339dc4/users/fa9b85c6c9484644994e69e53ce030d7/roles/49e47cd157d743f6905f1db07ec878a7` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:55:13.717763 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a3d854-8504-4691-99b0-ecf45c698bba admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b4132e03070403db1142a2454339dc4, user_id=fa9b85c6c9484644994e69e53ce030d7, role_id=49e47cd157d743f6905f1db07ec878a7)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:55:13.720412 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a3d854-8504-4691-99b0-ecf45c698bba admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:55:13.816585 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1368/2736] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Aug 24 06:55:13 2022] PUT /identity/v3/projects/4b4132e03070403db1142a2454339dc4/users/fa9b85c6c9484644994e69e53ce030d7/roles/49e47cd157d743f6905f1db07ec878a7 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 06:55:13.820518 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd226c6-1934-4b8a-93df-389cdbb4a95f admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:55:13.831143 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd226c6-1934-4b8a-93df-389cdbb4a95f admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:55:13.888815 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5b74b0-bf21-4a86-8078-04f94dc895bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:55:13.889422 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5b74b0-bf21-4a86-8078-04f94dc895bd admin admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:55:13.889722 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5b74b0-bf21-4a86-8078-04f94dc895bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:55:13.889994 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5b74b0-bf21-4a86-8078-04f94dc895bd admin admin] PATH_INFO: `/v3/roles` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:55:13.890722 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5b74b0-bf21-4a86-8078-04f94dc895bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:55:13.892205 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5b74b0-bf21-4a86-8078-04f94dc895bd admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:55:13.902608 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1369/2737] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Aug 24 06:55:13 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:55:13.906817 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a3d854-8504-4691-99b0-ecf45c698bba admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:55:13.917818 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a3d854-8504-4691-99b0-ecf45c698bba admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:55:13.922448 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb696b81-a270-4d88-ba8e-3c76d0284f5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:55:13.923087 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb696b81-a270-4d88-ba8e-3c76d0284f5f admin admin] REQUEST_METHOD: `PUT` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:55:13.923366 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb696b81-a270-4d88-ba8e-3c76d0284f5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:55:13.923640 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb696b81-a270-4d88-ba8e-3c76d0284f5f admin admin] PATH_INFO: `/v3/projects/4b4132e03070403db1142a2454339dc4/users/fa9b85c6c9484644994e69e53ce030d7/roles/49e47cd157d743f6905f1db07ec878a7` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:55:13.925606 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb696b81-a270-4d88-ba8e-3c76d0284f5f admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b4132e03070403db1142a2454339dc4, user_id=fa9b85c6c9484644994e69e53ce030d7, role_id=49e47cd157d743f6905f1db07ec878a7)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:55:13.928245 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb696b81-a270-4d88-ba8e-3c76d0284f5f admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:55:13.938516 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1369/2738] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Aug 24 06:55:13 2022] PUT /identity/v3/projects/4b4132e03070403db1142a2454339dc4/users/fa9b85c6c9484644994e69e53ce030d7/roles/49e47cd157d743f6905f1db07ec878a7 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 06:55:13.948276 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef2352f-2f22-4247-8801-f5ec613a720c None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:55:13.948591 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef2352f-2f22-4247-8801-f5ec613a720c None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:55:13.948868 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef2352f-2f22-4247-8801-f5ec613a720c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:55:13.976229 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-7ef2352f-2f22-4247-8801-f5ec613a720c None None] MFA Rules not processed for user `fa9b85c6c9484644994e69e53ce030d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:55:13.977535 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-7ef2352f-2f22-4247-8801-f5ec613a720c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:55:14.043728 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1370/2739] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:55:13 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:55:14.054163 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb696b81-a270-4d88-ba8e-3c76d0284f5f admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:55:14.055893 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-bb696b81-a270-4d88-ba8e-3c76d0284f5f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:55:14.114170 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb696b81-a270-4d88-ba8e-3c76d0284f5f admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:55:14.117854 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce814d83-a955-4955-adc1-6858fa16eec1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c4def55ac0e48b29836bc5c64bc7402', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:55:14.118483 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce814d83-a955-4955-adc1-6858fa16eec1 service cinder] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:55:14.118767 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce814d83-a955-4955-adc1-6858fa16eec1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:55:14.119043 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce814d83-a955-4955-adc1-6858fa16eec1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:55:14.128174 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce814d83-a955-4955-adc1-6858fa16eec1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:55:14.129645 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce814d83-a955-4955-adc1-6858fa16eec1 service cinder] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:55:14.139523 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1370/2740] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:55:14 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:55:14.187874 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef2352f-2f22-4247-8801-f5ec613a720c None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:55:14.189296 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-7ef2352f-2f22-4247-8801-f5ec613a720c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:55:14.201839 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef2352f-2f22-4247-8801-f5ec613a720c None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:55:14.205465 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d39383-fdc7-45b4-b7f3-773e9370f86d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c4def55ac0e48b29836bc5c64bc7402', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:55:14.206058 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d39383-fdc7-45b4-b7f3-773e9370f86d service cinder] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:55:14.206332 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d39383-fdc7-45b4-b7f3-773e9370f86d service cinder] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:55:14.206601 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d39383-fdc7-45b4-b7f3-773e9370f86d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:55:14.219477 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:55:14.264546 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d39383-fdc7-45b4-b7f3-773e9370f86d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:55:14.266008 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d39383-fdc7-45b4-b7f3-773e9370f86d service cinder] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:55:14.274790 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1371/2741] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:55:14 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:55:19.138276 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce814d83-a955-4955-adc1-6858fa16eec1 service cinder] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:55:19.188205 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:55:19.199859 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce814d83-a955-4955-adc1-6858fa16eec1 service cinder] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:55:19.204367 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384544c9-1d02-40c3-aa72-4322fd2ba4a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:55:19.205014 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384544c9-1d02-40c3-aa72-4322fd2ba4a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:55:19.205293 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384544c9-1d02-40c3-aa72-4322fd2ba4a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:55:19.205566 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384544c9-1d02-40c3-aa72-4322fd2ba4a1 admin admin] PATH_INFO: `/v3/users/11ebb079e2474f0c908069edab2f6a68` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:55:19.206668 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384544c9-1d02-40c3-aa72-4322fd2ba4a1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=11ebb079e2474f0c908069edab2f6a68)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:55:19.208678 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384544c9-1d02-40c3-aa72-4322fd2ba4a1 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:55:19.408600 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-384544c9-1d02-40c3-aa72-4322fd2ba4a1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '11ebb079e2474f0c908069edab2f6a68'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:55:19.415357 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-384544c9-1d02-40c3-aa72-4322fd2ba4a1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '11ebb079e2474f0c908069edab2f6a68'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:55:19.416149 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-384544c9-1d02-40c3-aa72-4322fd2ba4a1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '11ebb079e2474f0c908069edab2f6a68'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:55:19.491615 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-384544c9-1d02-40c3-aa72-4322fd2ba4a1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '11ebb079e2474f0c908069edab2f6a68'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:55:19.504962 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-384544c9-1d02-40c3-aa72-4322fd2ba4a1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '11ebb079e2474f0c908069edab2f6a68'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:55:19.511612 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1371/2742] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Aug 24 06:55:19 2022] DELETE /identity/v3/users/11ebb079e2474f0c908069edab2f6a68 => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:55:19.516271 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad22dba-26a2-4e36-8d8f-8634ee30a56f None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:55:19.516793 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad22dba-26a2-4e36-8d8f-8634ee30a56f None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:55:19.516920 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad22dba-26a2-4e36-8d8f-8634ee30a56f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:55:19.535229 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-dad22dba-26a2-4e36-8d8f-8634ee30a56f None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:55:19.536456 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-dad22dba-26a2-4e36-8d8f-8634ee30a56f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:55:19.595553 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1372/2743] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:55:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:55:19.605990 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384544c9-1d02-40c3-aa72-4322fd2ba4a1 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:55:19.607459 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-384544c9-1d02-40c3-aa72-4322fd2ba4a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:55:19.663313 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384544c9-1d02-40c3-aa72-4322fd2ba4a1 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:55:19.667826 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec444593-5124-41e7-a52e-b42a2f99ec91 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bb6c79db75e4d169331c02133d1ec32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:55:19.668331 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec444593-5124-41e7-a52e-b42a2f99ec91 service neutron] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:55:19.668719 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec444593-5124-41e7-a52e-b42a2f99ec91 service neutron] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:55:19.668914 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec444593-5124-41e7-a52e-b42a2f99ec91 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:55:19.678368 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec444593-5124-41e7-a52e-b42a2f99ec91 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:55:19.679801 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec444593-5124-41e7-a52e-b42a2f99ec91 service neutron] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:55:19.692072 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1372/2744] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:55:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:55:20.120287 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad22dba-26a2-4e36-8d8f-8634ee30a56f None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:55:20.128257 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-dad22dba-26a2-4e36-8d8f-8634ee30a56f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:55:20.136649 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad22dba-26a2-4e36-8d8f-8634ee30a56f None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:55:20.143169 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0cc068-2991-4300-a994-59cdf6526132 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:55:20.144324 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0cc068-2991-4300-a994-59cdf6526132 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:55:20.144933 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0cc068-2991-4300-a994-59cdf6526132 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:55:20.146382 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0cc068-2991-4300-a994-59cdf6526132 admin admin] PATH_INFO: `/v3/projects/530037605bf74e36a19e7459aa5c03c9` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:55:20.148155 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0cc068-2991-4300-a994-59cdf6526132 admin admin] RBAC: Authorizing `identity:delete_project(project_id=530037605bf74e36a19e7459aa5c03c9)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:55:20.157030 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0cc068-2991-4300-a994-59cdf6526132 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:55:20.238165 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-cb0cc068-2991-4300-a994-59cdf6526132 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '530037605bf74e36a19e7459aa5c03c9', 'request_id': 'req-cb0cc068-2991-4300-a994-59cdf6526132'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:55:20.243549 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-cb0cc068-2991-4300-a994-59cdf6526132 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '530037605bf74e36a19e7459aa5c03c9', 'request_id': 'req-cb0cc068-2991-4300-a994-59cdf6526132'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:55:20.249852 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-cb0cc068-2991-4300-a994-59cdf6526132 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '530037605bf74e36a19e7459aa5c03c9', 'request_id': 'req-cb0cc068-2991-4300-a994-59cdf6526132'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:55:20.313630 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-cb0cc068-2991-4300-a994-59cdf6526132 admin admin] The token cache is being invalidate because project 530037605bf74e36a19e7459aa5c03c9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81851) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:55:20.313917 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-cb0cc068-2991-4300-a994-59cdf6526132 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:55:20.314547 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-cb0cc068-2991-4300-a994-59cdf6526132 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:55:20.316052 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1373/2745] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Aug 24 06:55:20 2022] DELETE /identity/v3/projects/530037605bf74e36a19e7459aa5c03c9 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:55:20.320488 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec444593-5124-41e7-a52e-b42a2f99ec91 service neutron] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:55:20.331112 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec444593-5124-41e7-a52e-b42a2f99ec91 service neutron] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:55:20.387229 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e962e8-2b2c-444f-a7d5-8c0fcd14d9ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:55:20.387853 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e962e8-2b2c-444f-a7d5-8c0fcd14d9ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:55:20.388130 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e962e8-2b2c-444f-a7d5-8c0fcd14d9ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:55:20.388401 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e962e8-2b2c-444f-a7d5-8c0fcd14d9ec admin admin] PATH_INFO: `/v3/users/fa9b85c6c9484644994e69e53ce030d7` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:55:20.389534 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e962e8-2b2c-444f-a7d5-8c0fcd14d9ec admin admin] RBAC: Authorizing `identity:delete_user(user_id=fa9b85c6c9484644994e69e53ce030d7)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:55:20.391454 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e962e8-2b2c-444f-a7d5-8c0fcd14d9ec admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:55:20.405389 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:55:20.602632 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-22e962e8-2b2c-444f-a7d5-8c0fcd14d9ec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fa9b85c6c9484644994e69e53ce030d7'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:55:20.610878 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-22e962e8-2b2c-444f-a7d5-8c0fcd14d9ec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fa9b85c6c9484644994e69e53ce030d7'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:55:20.611561 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-22e962e8-2b2c-444f-a7d5-8c0fcd14d9ec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fa9b85c6c9484644994e69e53ce030d7'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:55:20.697651 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-22e962e8-2b2c-444f-a7d5-8c0fcd14d9ec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fa9b85c6c9484644994e69e53ce030d7'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:55:20.710627 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-22e962e8-2b2c-444f-a7d5-8c0fcd14d9ec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fa9b85c6c9484644994e69e53ce030d7'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:55:20.717223 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1373/2746] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Aug 24 06:55:20 2022] DELETE /identity/v3/users/fa9b85c6c9484644994e69e53ce030d7 => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:55:21.151995 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0cc068-2991-4300-a994-59cdf6526132 admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:55:21.153663 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-cb0cc068-2991-4300-a994-59cdf6526132 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:55:21.173106 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0cc068-2991-4300-a994-59cdf6526132 admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:55:21.211057 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:55:21.235482 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35db7357-bb9e-4d23-806f-4615fa5bdcc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:55:21.236107 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35db7357-bb9e-4d23-806f-4615fa5bdcc0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:55:21.236418 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35db7357-bb9e-4d23-806f-4615fa5bdcc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:55:21.236721 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35db7357-bb9e-4d23-806f-4615fa5bdcc0 admin admin] PATH_INFO: `/v3/projects/4b4132e03070403db1142a2454339dc4` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:55:21.237863 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35db7357-bb9e-4d23-806f-4615fa5bdcc0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4b4132e03070403db1142a2454339dc4)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:55:21.242031 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35db7357-bb9e-4d23-806f-4615fa5bdcc0 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:55:21.341803 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-35db7357-bb9e-4d23-806f-4615fa5bdcc0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4b4132e03070403db1142a2454339dc4', 'request_id': 'req-35db7357-bb9e-4d23-806f-4615fa5bdcc0'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:55:21.347223 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-35db7357-bb9e-4d23-806f-4615fa5bdcc0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4b4132e03070403db1142a2454339dc4', 'request_id': 'req-35db7357-bb9e-4d23-806f-4615fa5bdcc0'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:55:21.354156 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-35db7357-bb9e-4d23-806f-4615fa5bdcc0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4b4132e03070403db1142a2454339dc4', 'request_id': 'req-35db7357-bb9e-4d23-806f-4615fa5bdcc0'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:55:22.060839 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-35db7357-bb9e-4d23-806f-4615fa5bdcc0 admin admin] The token cache is being invalidate because project 4b4132e03070403db1142a2454339dc4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81851) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:55:22.061040 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-35db7357-bb9e-4d23-806f-4615fa5bdcc0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:55:22.061671 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-35db7357-bb9e-4d23-806f-4615fa5bdcc0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:55:22.063190 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1374/2747] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Aug 24 06:55:21 2022] DELETE /identity/v3/projects/4b4132e03070403db1142a2454339dc4 => generated 0 bytes in 912 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:55:22.068529 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132b1d30-d181-4ab8-bf05-7fd1cb1a7202 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:55:22.069009 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132b1d30-d181-4ab8-bf05-7fd1cb1a7202 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:55:22.069097 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132b1d30-d181-4ab8-bf05-7fd1cb1a7202 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:55:22.089554 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-132b1d30-d181-4ab8-bf05-7fd1cb1a7202 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:55:22.090784 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-132b1d30-d181-4ab8-bf05-7fd1cb1a7202 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:55:22.148641 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1374/2748] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:55:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:55:22.156778 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3a4a4a-4063-4f62-bac6-bf821ed110d6 None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:55:22.157277 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3a4a4a-4063-4f62-bac6-bf821ed110d6 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:55:22.157373 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3a4a4a-4063-4f62-bac6-bf821ed110d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:55:22.177277 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-7b3a4a4a-4063-4f62-bac6-bf821ed110d6 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:55:22.179046 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-7b3a4a4a-4063-4f62-bac6-bf821ed110d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:55:22.186924 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1375/2749] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:55:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:55:22.191562 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132b1d30-d181-4ab8-bf05-7fd1cb1a7202 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:55:22.201261 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132b1d30-d181-4ab8-bf05-7fd1cb1a7202 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:55:22.206228 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc07e32-5466-4480-b89a-dbb638b8344d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:55:22.206833 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc07e32-5466-4480-b89a-dbb638b8344d admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:55:22.207114 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc07e32-5466-4480-b89a-dbb638b8344d admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:55:22.207407 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc07e32-5466-4480-b89a-dbb638b8344d admin admin] PATH_INFO: `/v3/domains` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:55:22.208055 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc07e32-5466-4480-b89a-dbb638b8344d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:55:22.208361 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc07e32-5466-4480-b89a-dbb638b8344d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:55:22.209995 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc07e32-5466-4480-b89a-dbb638b8344d admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:55:22.222217 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1375/2750] 10.222.0.62 () {58 vars in 1153 bytes} [Wed Aug 24 06:55:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:55:22.226545 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3a4a4a-4063-4f62-bac6-bf821ed110d6 None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:55:22.236516 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3a4a4a-4063-4f62-bac6-bf821ed110d6 None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:55:22.241347 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bb77e6-f302-40eb-9ff0-4b03a5946c40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:55:22.242016 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bb77e6-f302-40eb-9ff0-4b03a5946c40 admin admin] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:55:22.242420 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bb77e6-f302-40eb-9ff0-4b03a5946c40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:55:22.242703 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bb77e6-f302-40eb-9ff0-4b03a5946c40 admin admin] PATH_INFO: `/v3/projects` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:55:22.243354 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bb77e6-f302-40eb-9ff0-4b03a5946c40 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:55:22.245309 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bb77e6-f302-40eb-9ff0-4b03a5946c40 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:55:22.331259 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1376/2751] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Aug 24 06:55:22 2022] POST /identity/v3/projects => generated 381 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:55:22.335633 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc07e32-5466-4480-b89a-dbb638b8344d admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:55:22.346039 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc07e32-5466-4480-b89a-dbb638b8344d admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:55:22.401521 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54413fc1-31af-4127-bfcf-ccb1cffa0b2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:55:22.402127 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54413fc1-31af-4127-bfcf-ccb1cffa0b2b admin admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:55:22.402412 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54413fc1-31af-4127-bfcf-ccb1cffa0b2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:55:22.402687 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54413fc1-31af-4127-bfcf-ccb1cffa0b2b admin admin] PATH_INFO: `/v3/users` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:55:22.406951 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54413fc1-31af-4127-bfcf-ccb1cffa0b2b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:55:22.406951 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54413fc1-31af-4127-bfcf-ccb1cffa0b2b admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:55:22.509369 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1376/2752] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Aug 24 06:55:22 2022] POST /identity/v3/users => generated 291 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:55:22.513627 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bb77e6-f302-40eb-9ff0-4b03a5946c40 admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:55:22.524078 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bb77e6-f302-40eb-9ff0-4b03a5946c40 admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:55:22.528780 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e01cc3-80d2-47d7-9d2b-b4d5d2456087 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:55:22.529360 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e01cc3-80d2-47d7-9d2b-b4d5d2456087 admin admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:55:22.529644 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e01cc3-80d2-47d7-9d2b-b4d5d2456087 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:55:22.529920 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e01cc3-80d2-47d7-9d2b-b4d5d2456087 admin admin] PATH_INFO: `/v3/roles` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:55:22.530562 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e01cc3-80d2-47d7-9d2b-b4d5d2456087 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:55:22.532158 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e01cc3-80d2-47d7-9d2b-b4d5d2456087 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:55:22.542407 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1377/2753] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Aug 24 06:55:22 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 06:55:22.546267 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54413fc1-31af-4127-bfcf-ccb1cffa0b2b admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:55:22.556796 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54413fc1-31af-4127-bfcf-ccb1cffa0b2b admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:55:22.561377 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a38f50-9eb9-49e9-9e61-f11d495ed465 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:55:22.561963 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a38f50-9eb9-49e9-9e61-f11d495ed465 admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:55:22.562247 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a38f50-9eb9-49e9-9e61-f11d495ed465 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:55:22.562551 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a38f50-9eb9-49e9-9e61-f11d495ed465 admin admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:55:22.563180 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a38f50-9eb9-49e9-9e61-f11d495ed465 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:55:22.564811 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a38f50-9eb9-49e9-9e61-f11d495ed465 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:55:22.567147 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:55:22.576170 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1377/2754] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Aug 24 06:55:22 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:55:22.580675 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e01cc3-80d2-47d7-9d2b-b4d5d2456087 admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:55:22.592872 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e01cc3-80d2-47d7-9d2b-b4d5d2456087 admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:55:22.595449 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2207b295-b135-4768-bb9e-c26edc82ee62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:55:22.596089 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2207b295-b135-4768-bb9e-c26edc82ee62 admin admin] REQUEST_METHOD: `PUT` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:55:22.596366 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2207b295-b135-4768-bb9e-c26edc82ee62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:55:22.596672 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2207b295-b135-4768-bb9e-c26edc82ee62 admin admin] PATH_INFO: `/v3/projects/303e09bd699c4f8c88cef721f759910e/users/782aafc3cc8c450b87ab6023184c66f3/roles/580aa7dbb38d4cff8f9d1bb8983f99ef` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:55:22.613160 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2207b295-b135-4768-bb9e-c26edc82ee62 admin admin] RBAC: Authorizing `identity:create_grant(project_id=303e09bd699c4f8c88cef721f759910e, user_id=782aafc3cc8c450b87ab6023184c66f3, role_id=580aa7dbb38d4cff8f9d1bb8983f99ef)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:55:22.615841 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2207b295-b135-4768-bb9e-c26edc82ee62 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:55:22.709205 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1378/2755] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Aug 24 06:55:22 2022] PUT /identity/v3/projects/303e09bd699c4f8c88cef721f759910e/users/782aafc3cc8c450b87ab6023184c66f3/roles/580aa7dbb38d4cff8f9d1bb8983f99ef => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 06:55:22.720919 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d88ffe-ef07-467d-80b2-1e928ffefd39 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:55:22.721205 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d88ffe-ef07-467d-80b2-1e928ffefd39 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:55:22.721502 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d88ffe-ef07-467d-80b2-1e928ffefd39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:55:22.750120 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-d0d88ffe-ef07-467d-80b2-1e928ffefd39 None None] MFA Rules not processed for user `782aafc3cc8c450b87ab6023184c66f3`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:55:22.751342 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-d0d88ffe-ef07-467d-80b2-1e928ffefd39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:55:22.814822 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1378/2756] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:55:22 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:55:22.819904 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2207b295-b135-4768-bb9e-c26edc82ee62 admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:55:22.884706 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2207b295-b135-4768-bb9e-c26edc82ee62 admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:55:22.889551 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b276a700-1dc1-493f-93e6-79b3264bf937 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:55:22.890216 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b276a700-1dc1-493f-93e6-79b3264bf937 admin admin] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:55:22.890496 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b276a700-1dc1-493f-93e6-79b3264bf937 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:55:22.890774 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b276a700-1dc1-493f-93e6-79b3264bf937 admin admin] PATH_INFO: `/v3/projects` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:55:22.891391 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b276a700-1dc1-493f-93e6-79b3264bf937 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:55:22.893331 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b276a700-1dc1-493f-93e6-79b3264bf937 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:55:22.903366 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:55:22.990436 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1379/2757] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Aug 24 06:55:22 2022] POST /identity/v3/projects => generated 381 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:55:22.995027 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d88ffe-ef07-467d-80b2-1e928ffefd39 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:55:23.005805 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d88ffe-ef07-467d-80b2-1e928ffefd39 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:55:23.062129 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95425cb6-0d52-4426-b096-8a7a2bb8afa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:55:23.062739 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95425cb6-0d52-4426-b096-8a7a2bb8afa0 admin admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:55:23.063018 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95425cb6-0d52-4426-b096-8a7a2bb8afa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:55:23.063288 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95425cb6-0d52-4426-b096-8a7a2bb8afa0 admin admin] PATH_INFO: `/v3/users` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:55:23.063917 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95425cb6-0d52-4426-b096-8a7a2bb8afa0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:55:23.065852 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95425cb6-0d52-4426-b096-8a7a2bb8afa0 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:55:23.176281 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1379/2758] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Aug 24 06:55:22 2022] POST /identity/v3/users => generated 297 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 06:55:23.180120 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b276a700-1dc1-493f-93e6-79b3264bf937 admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:55:23.191500 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b276a700-1dc1-493f-93e6-79b3264bf937 admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:55:23.196153 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52442df9-09e0-41f1-b401-3402e4198ca2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:55:23.196792 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52442df9-09e0-41f1-b401-3402e4198ca2 admin admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:55:23.197071 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52442df9-09e0-41f1-b401-3402e4198ca2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:55:23.197372 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52442df9-09e0-41f1-b401-3402e4198ca2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:55:23.198012 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52442df9-09e0-41f1-b401-3402e4198ca2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:55:23.199749 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52442df9-09e0-41f1-b401-3402e4198ca2 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:55:23.210098 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1380/2759] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Aug 24 06:55:23 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:55:23.214390 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95425cb6-0d52-4426-b096-8a7a2bb8afa0 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:55:23.224332 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95425cb6-0d52-4426-b096-8a7a2bb8afa0 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:55:23.229449 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fd3d11-486e-433d-b76c-d50fa6a02051 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:55:23.230094 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fd3d11-486e-433d-b76c-d50fa6a02051 admin admin] REQUEST_METHOD: `PUT` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:55:23.230372 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fd3d11-486e-433d-b76c-d50fa6a02051 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:55:23.230644 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fd3d11-486e-433d-b76c-d50fa6a02051 admin admin] PATH_INFO: `/v3/projects/4f1208c1f7bd4568b6816b5513989f93/users/cfa954c73da54414b24efc277201e4f3/roles/49e47cd157d743f6905f1db07ec878a7` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:55:23.247174 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3fd3d11-486e-433d-b76c-d50fa6a02051 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f1208c1f7bd4568b6816b5513989f93, user_id=cfa954c73da54414b24efc277201e4f3, role_id=49e47cd157d743f6905f1db07ec878a7)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:55:23.249900 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3fd3d11-486e-433d-b76c-d50fa6a02051 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:55:23.335767 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1380/2760] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Aug 24 06:55:23 2022] PUT /identity/v3/projects/4f1208c1f7bd4568b6816b5513989f93/users/cfa954c73da54414b24efc277201e4f3/roles/49e47cd157d743f6905f1db07ec878a7 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:55:23.340100 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52442df9-09e0-41f1-b401-3402e4198ca2 admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:55:23.350278 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52442df9-09e0-41f1-b401-3402e4198ca2 admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:55:23.405952 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9dc8b8f-08ea-433a-b2da-8bb121ea8b3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:55:23.406563 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dc8b8f-08ea-433a-b2da-8bb121ea8b3b admin admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:55:23.406839 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dc8b8f-08ea-433a-b2da-8bb121ea8b3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:55:23.407122 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dc8b8f-08ea-433a-b2da-8bb121ea8b3b admin admin] PATH_INFO: `/v3/roles` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:55:23.407760 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9dc8b8f-08ea-433a-b2da-8bb121ea8b3b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:55:23.409389 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9dc8b8f-08ea-433a-b2da-8bb121ea8b3b admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:55:23.419573 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1381/2761] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Aug 24 06:55:23 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:55:23.423752 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fd3d11-486e-433d-b76c-d50fa6a02051 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:55:23.434844 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fd3d11-486e-433d-b76c-d50fa6a02051 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:55:23.439378 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c459b27-020a-4e43-97ec-f7a47f43bd96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:55:23.440019 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c459b27-020a-4e43-97ec-f7a47f43bd96 admin admin] REQUEST_METHOD: `PUT` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:55:23.440295 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c459b27-020a-4e43-97ec-f7a47f43bd96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:55:23.440600 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c459b27-020a-4e43-97ec-f7a47f43bd96 admin admin] PATH_INFO: `/v3/projects/4f1208c1f7bd4568b6816b5513989f93/users/cfa954c73da54414b24efc277201e4f3/roles/49e47cd157d743f6905f1db07ec878a7` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:55:23.442752 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c459b27-020a-4e43-97ec-f7a47f43bd96 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f1208c1f7bd4568b6816b5513989f93, user_id=cfa954c73da54414b24efc277201e4f3, role_id=49e47cd157d743f6905f1db07ec878a7)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:55:23.445418 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c459b27-020a-4e43-97ec-f7a47f43bd96 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:55:23.455374 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1381/2762] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Aug 24 06:55:23 2022] PUT /identity/v3/projects/4f1208c1f7bd4568b6816b5513989f93/users/cfa954c73da54414b24efc277201e4f3/roles/49e47cd157d743f6905f1db07ec878a7 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:55:23.465273 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d44aef-91b1-49d2-bb26-24a4e4419e8b None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:55:23.465568 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d44aef-91b1-49d2-bb26-24a4e4419e8b None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:55:23.465851 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d44aef-91b1-49d2-bb26-24a4e4419e8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:55:23.494132 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-81d44aef-91b1-49d2-bb26-24a4e4419e8b None None] MFA Rules not processed for user `cfa954c73da54414b24efc277201e4f3`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:55:23.495343 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-81d44aef-91b1-49d2-bb26-24a4e4419e8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:55:23.562419 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1382/2763] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:55:23 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:55:23.574152 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c459b27-020a-4e43-97ec-f7a47f43bd96 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:55:23.575898 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-1c459b27-020a-4e43-97ec-f7a47f43bd96 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:55:23.620423 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:55:23.633530 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c459b27-020a-4e43-97ec-f7a47f43bd96 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:55:23.637167 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b2954d-ae6e-4713-977e-0a153dbd0e26 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c4def55ac0e48b29836bc5c64bc7402', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:55:23.637760 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b2954d-ae6e-4713-977e-0a153dbd0e26 service cinder] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:55:23.638035 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b2954d-ae6e-4713-977e-0a153dbd0e26 service cinder] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:55:23.638304 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b2954d-ae6e-4713-977e-0a153dbd0e26 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:55:23.647107 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b2954d-ae6e-4713-977e-0a153dbd0e26 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:55:23.648555 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b2954d-ae6e-4713-977e-0a153dbd0e26 service cinder] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:55:23.659115 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1382/2764] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:55:23 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:55:23.812772 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d44aef-91b1-49d2-bb26-24a4e4419e8b None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:55:23.823150 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d44aef-91b1-49d2-bb26-24a4e4419e8b None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:55:23.827321 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7528b783-6193-4a6d-b9b9-e5d488f60a95 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c4def55ac0e48b29836bc5c64bc7402', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:55:23.827867 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7528b783-6193-4a6d-b9b9-e5d488f60a95 service cinder] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:55:23.828146 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7528b783-6193-4a6d-b9b9-e5d488f60a95 service cinder] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:55:23.828426 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7528b783-6193-4a6d-b9b9-e5d488f60a95 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:55:23.885502 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7528b783-6193-4a6d-b9b9-e5d488f60a95 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:55:23.886951 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7528b783-6193-4a6d-b9b9-e5d488f60a95 service cinder] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:55:23.891345 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:55:23.897097 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1383/2765] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:55:23 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:55:24.329528 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b2954d-ae6e-4713-977e-0a153dbd0e26 service cinder] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:55:24.392246 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b2954d-ae6e-4713-977e-0a153dbd0e26 service cinder] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:55:24.396720 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038905df-4ed1-4a8f-9c5d-4ba955439d9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:55:24.397338 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038905df-4ed1-4a8f-9c5d-4ba955439d9a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:55:24.397611 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038905df-4ed1-4a8f-9c5d-4ba955439d9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:55:24.397879 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038905df-4ed1-4a8f-9c5d-4ba955439d9a admin admin] PATH_INFO: `/v3/users/782aafc3cc8c450b87ab6023184c66f3` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:55:24.399060 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038905df-4ed1-4a8f-9c5d-4ba955439d9a admin admin] RBAC: Authorizing `identity:delete_user(user_id=782aafc3cc8c450b87ab6023184c66f3)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:55:24.401586 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038905df-4ed1-4a8f-9c5d-4ba955439d9a admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:55:24.649627 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-038905df-4ed1-4a8f-9c5d-4ba955439d9a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '782aafc3cc8c450b87ab6023184c66f3'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:55:24.658261 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-038905df-4ed1-4a8f-9c5d-4ba955439d9a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '782aafc3cc8c450b87ab6023184c66f3'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:55:24.658977 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-038905df-4ed1-4a8f-9c5d-4ba955439d9a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '782aafc3cc8c450b87ab6023184c66f3'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:55:24.731806 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-038905df-4ed1-4a8f-9c5d-4ba955439d9a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '782aafc3cc8c450b87ab6023184c66f3'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:55:24.744787 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-038905df-4ed1-4a8f-9c5d-4ba955439d9a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '782aafc3cc8c450b87ab6023184c66f3'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:55:24.751104 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1383/2766] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Aug 24 06:55:24 2022] DELETE /identity/v3/users/782aafc3cc8c450b87ab6023184c66f3 => generated 0 bytes in 422 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:55:24.755843 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16eabc37-6b27-4cf7-9f0e-accbe56fdb28 None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:55:24.756411 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16eabc37-6b27-4cf7-9f0e-accbe56fdb28 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:55:24.756411 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16eabc37-6b27-4cf7-9f0e-accbe56fdb28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:55:24.776642 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-16eabc37-6b27-4cf7-9f0e-accbe56fdb28 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:55:24.777880 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-16eabc37-6b27-4cf7-9f0e-accbe56fdb28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:55:24.841150 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1384/2767] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:55:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:55:24.850759 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038905df-4ed1-4a8f-9c5d-4ba955439d9a admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:55:24.852179 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-038905df-4ed1-4a8f-9c5d-4ba955439d9a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:55:24.909224 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038905df-4ed1-4a8f-9c5d-4ba955439d9a admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:55:24.912700 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b68b3dd-e9bf-49f2-bc2b-5872d86d702d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bb6c79db75e4d169331c02133d1ec32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:55:24.913313 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b68b3dd-e9bf-49f2-bc2b-5872d86d702d service neutron] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:55:24.913591 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b68b3dd-e9bf-49f2-bc2b-5872d86d702d service neutron] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:55:24.913860 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b68b3dd-e9bf-49f2-bc2b-5872d86d702d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:55:24.922794 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b68b3dd-e9bf-49f2-bc2b-5872d86d702d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:55:24.924219 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b68b3dd-e9bf-49f2-bc2b-5872d86d702d service neutron] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:55:24.933811 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1384/2768] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:55:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:55:25.584885 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16eabc37-6b27-4cf7-9f0e-accbe56fdb28 None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:55:25.586296 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-16eabc37-6b27-4cf7-9f0e-accbe56fdb28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:55:25.601142 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16eabc37-6b27-4cf7-9f0e-accbe56fdb28 None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:55:25.606594 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50271ae6-100e-4f08-a6c4-f5d12a45e239 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:55:25.607251 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50271ae6-100e-4f08-a6c4-f5d12a45e239 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:55:25.607572 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50271ae6-100e-4f08-a6c4-f5d12a45e239 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:55:25.607890 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50271ae6-100e-4f08-a6c4-f5d12a45e239 admin admin] PATH_INFO: `/v3/projects/303e09bd699c4f8c88cef721f759910e` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:55:25.609132 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50271ae6-100e-4f08-a6c4-f5d12a45e239 admin admin] RBAC: Authorizing `identity:delete_project(project_id=303e09bd699c4f8c88cef721f759910e)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:55:25.611052 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50271ae6-100e-4f08-a6c4-f5d12a45e239 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:55:25.703193 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-50271ae6-100e-4f08-a6c4-f5d12a45e239 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '303e09bd699c4f8c88cef721f759910e', 'request_id': 'req-50271ae6-100e-4f08-a6c4-f5d12a45e239'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:55:25.708814 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-50271ae6-100e-4f08-a6c4-f5d12a45e239 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '303e09bd699c4f8c88cef721f759910e', 'request_id': 'req-50271ae6-100e-4f08-a6c4-f5d12a45e239'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:55:25.715174 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-50271ae6-100e-4f08-a6c4-f5d12a45e239 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '303e09bd699c4f8c88cef721f759910e', 'request_id': 'req-50271ae6-100e-4f08-a6c4-f5d12a45e239'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:55:25.788325 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-50271ae6-100e-4f08-a6c4-f5d12a45e239 admin admin] The token cache is being invalidate because project 303e09bd699c4f8c88cef721f759910e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81851) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:55:25.789022 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-50271ae6-100e-4f08-a6c4-f5d12a45e239 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:55:25.789353 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-50271ae6-100e-4f08-a6c4-f5d12a45e239 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:55:25.791531 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1385/2769] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Aug 24 06:55:25 2022] DELETE /identity/v3/projects/303e09bd699c4f8c88cef721f759910e => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:55:25.795521 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b68b3dd-e9bf-49f2-bc2b-5872d86d702d service neutron] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:55:25.801661 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:55:25.805312 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b68b3dd-e9bf-49f2-bc2b-5872d86d702d service neutron] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:55:25.864095 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975f3cc1-39c4-4aec-b68b-6f31ed035b86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:55:25.864768 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975f3cc1-39c4-4aec-b68b-6f31ed035b86 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:55:25.865047 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975f3cc1-39c4-4aec-b68b-6f31ed035b86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:55:25.865318 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975f3cc1-39c4-4aec-b68b-6f31ed035b86 admin admin] PATH_INFO: `/v3/users/cfa954c73da54414b24efc277201e4f3` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:55:25.866428 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975f3cc1-39c4-4aec-b68b-6f31ed035b86 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cfa954c73da54414b24efc277201e4f3)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:55:25.868374 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975f3cc1-39c4-4aec-b68b-6f31ed035b86 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:55:26.037039 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-975f3cc1-39c4-4aec-b68b-6f31ed035b86 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cfa954c73da54414b24efc277201e4f3'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:55:26.045291 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-975f3cc1-39c4-4aec-b68b-6f31ed035b86 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cfa954c73da54414b24efc277201e4f3'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:55:26.046015 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-975f3cc1-39c4-4aec-b68b-6f31ed035b86 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cfa954c73da54414b24efc277201e4f3'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:55:26.097024 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-975f3cc1-39c4-4aec-b68b-6f31ed035b86 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cfa954c73da54414b24efc277201e4f3'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:55:26.109761 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-975f3cc1-39c4-4aec-b68b-6f31ed035b86 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cfa954c73da54414b24efc277201e4f3'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:55:26.117784 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1385/2770] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Aug 24 06:55:25 2022] DELETE /identity/v3/users/cfa954c73da54414b24efc277201e4f3 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:55:26.608806 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50271ae6-100e-4f08-a6c4-f5d12a45e239 admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:55:26.610413 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-50271ae6-100e-4f08-a6c4-f5d12a45e239 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:55:26.624227 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50271ae6-100e-4f08-a6c4-f5d12a45e239 admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:55:26.689809 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723b56e4-0fb0-4ce4-807c-88910a8db9e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:55:26.690442 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723b56e4-0fb0-4ce4-807c-88910a8db9e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:55:26.690719 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723b56e4-0fb0-4ce4-807c-88910a8db9e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:55:26.690988 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723b56e4-0fb0-4ce4-807c-88910a8db9e9 admin admin] PATH_INFO: `/v3/projects/4f1208c1f7bd4568b6816b5513989f93` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:55:26.692350 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723b56e4-0fb0-4ce4-807c-88910a8db9e9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4f1208c1f7bd4568b6816b5513989f93)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:55:26.694286 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723b56e4-0fb0-4ce4-807c-88910a8db9e9 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:55:26.719356 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:55:26.828928 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-723b56e4-0fb0-4ce4-807c-88910a8db9e9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4f1208c1f7bd4568b6816b5513989f93', 'request_id': 'req-723b56e4-0fb0-4ce4-807c-88910a8db9e9'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:55:26.834474 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-723b56e4-0fb0-4ce4-807c-88910a8db9e9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4f1208c1f7bd4568b6816b5513989f93', 'request_id': 'req-723b56e4-0fb0-4ce4-807c-88910a8db9e9'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:55:26.841179 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-723b56e4-0fb0-4ce4-807c-88910a8db9e9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4f1208c1f7bd4568b6816b5513989f93', 'request_id': 'req-723b56e4-0fb0-4ce4-807c-88910a8db9e9'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:55:26.952676 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-723b56e4-0fb0-4ce4-807c-88910a8db9e9 admin admin] The token cache is being invalidate because project 4f1208c1f7bd4568b6816b5513989f93 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81851) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:55:26.952983 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-723b56e4-0fb0-4ce4-807c-88910a8db9e9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:55:26.953590 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-723b56e4-0fb0-4ce4-807c-88910a8db9e9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:55:26.955111 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1386/2771] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Aug 24 06:55:26 2022] DELETE /identity/v3/projects/4f1208c1f7bd4568b6816b5513989f93 => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:55:26.959978 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537ec165-3cee-4c79-afe2-61bd99331c4e None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:55:26.967001 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537ec165-3cee-4c79-afe2-61bd99331c4e None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:55:26.967001 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537ec165-3cee-4c79-afe2-61bd99331c4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:55:26.980754 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-537ec165-3cee-4c79-afe2-61bd99331c4e None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:55:26.981950 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-537ec165-3cee-4c79-afe2-61bd99331c4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:55:27.038384 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1386/2772] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:55:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:55:27.047888 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63df9ab3-d294-4876-8607-6c632d41fa5d None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:55:27.048170 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63df9ab3-d294-4876-8607-6c632d41fa5d None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:55:27.048444 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63df9ab3-d294-4876-8607-6c632d41fa5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:55:27.069806 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-63df9ab3-d294-4876-8607-6c632d41fa5d None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:55:27.071017 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-63df9ab3-d294-4876-8607-6c632d41fa5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:55:27.080099 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1387/2773] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:55:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:55:27.089514 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537ec165-3cee-4c79-afe2-61bd99331c4e None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:55:27.094053 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537ec165-3cee-4c79-afe2-61bd99331c4e None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:55:27.098717 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd2fe1d-9315-4ede-94df-7592c49b0056 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:55:27.099321 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd2fe1d-9315-4ede-94df-7592c49b0056 admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:55:27.099598 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd2fe1d-9315-4ede-94df-7592c49b0056 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:55:27.099981 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd2fe1d-9315-4ede-94df-7592c49b0056 admin admin] PATH_INFO: `/v3/domains` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:55:27.100718 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd2fe1d-9315-4ede-94df-7592c49b0056 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:55:27.101025 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd2fe1d-9315-4ede-94df-7592c49b0056 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:55:27.103134 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd2fe1d-9315-4ede-94df-7592c49b0056 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:55:27.114717 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1387/2774] 10.222.0.62 () {58 vars in 1153 bytes} [Wed Aug 24 06:55:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:55:27.121727 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63df9ab3-d294-4876-8607-6c632d41fa5d None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:55:27.130883 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63df9ab3-d294-4876-8607-6c632d41fa5d None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:55:27.135495 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92981f12-b27c-4d0c-bd95-aef9bdd890a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:55:27.136119 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92981f12-b27c-4d0c-bd95-aef9bdd890a1 admin admin] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:55:27.136393 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92981f12-b27c-4d0c-bd95-aef9bdd890a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:55:27.136696 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92981f12-b27c-4d0c-bd95-aef9bdd890a1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:55:27.137320 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92981f12-b27c-4d0c-bd95-aef9bdd890a1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:55:27.139198 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92981f12-b27c-4d0c-bd95-aef9bdd890a1 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:55:27.259392 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1388/2775] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Aug 24 06:55:27 2022] POST /identity/v3/projects => generated 373 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 06:55:27.263664 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd2fe1d-9315-4ede-94df-7592c49b0056 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:55:27.276056 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd2fe1d-9315-4ede-94df-7592c49b0056 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:55:27.283894 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:55:27.332778 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c7c5ef-f1fe-4143-a165-d117b87acf18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:55:27.333391 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c7c5ef-f1fe-4143-a165-d117b87acf18 admin admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:55:27.333666 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c7c5ef-f1fe-4143-a165-d117b87acf18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:55:27.333932 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c7c5ef-f1fe-4143-a165-d117b87acf18 admin admin] PATH_INFO: `/v3/users` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:55:27.334586 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c7c5ef-f1fe-4143-a165-d117b87acf18 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:55:27.336559 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c7c5ef-f1fe-4143-a165-d117b87acf18 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:55:27.423266 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1388/2776] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Aug 24 06:55:27 2022] POST /identity/v3/users => generated 287 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:55:27.427455 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92981f12-b27c-4d0c-bd95-aef9bdd890a1 admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:55:27.437869 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92981f12-b27c-4d0c-bd95-aef9bdd890a1 admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:55:27.442685 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdfbd9a-5f12-457a-b274-aab778b4ae42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:55:27.443284 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdfbd9a-5f12-457a-b274-aab778b4ae42 admin admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:55:27.443564 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdfbd9a-5f12-457a-b274-aab778b4ae42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:55:27.444648 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdfbd9a-5f12-457a-b274-aab778b4ae42 admin admin] PATH_INFO: `/v3/roles` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:55:27.444648 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdfbd9a-5f12-457a-b274-aab778b4ae42 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:55:27.446223 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdfbd9a-5f12-457a-b274-aab778b4ae42 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:55:27.456898 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1389/2777] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Aug 24 06:55:27 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:55:27.463269 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c7c5ef-f1fe-4143-a165-d117b87acf18 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:55:27.471589 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c7c5ef-f1fe-4143-a165-d117b87acf18 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:55:27.476176 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c2439e-487a-40ae-aee1-4dd4f9dfeb5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:55:27.476805 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c2439e-487a-40ae-aee1-4dd4f9dfeb5d admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:55:27.477081 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c2439e-487a-40ae-aee1-4dd4f9dfeb5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:55:27.477348 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c2439e-487a-40ae-aee1-4dd4f9dfeb5d admin admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:55:27.478588 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c2439e-487a-40ae-aee1-4dd4f9dfeb5d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:55:27.480167 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c2439e-487a-40ae-aee1-4dd4f9dfeb5d admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:55:27.490672 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1389/2778] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Aug 24 06:55:27 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:55:27.494899 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdfbd9a-5f12-457a-b274-aab778b4ae42 admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:55:27.504782 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdfbd9a-5f12-457a-b274-aab778b4ae42 admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:55:27.509459 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b535bc7-ce22-4df5-987d-55fb7a64208b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:55:27.510064 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b535bc7-ce22-4df5-987d-55fb7a64208b admin admin] REQUEST_METHOD: `PUT` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:55:27.510341 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b535bc7-ce22-4df5-987d-55fb7a64208b admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:55:27.510613 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b535bc7-ce22-4df5-987d-55fb7a64208b admin admin] PATH_INFO: `/v3/projects/6383ffd740d947dba6cc24008d830ef5/users/51dce3e1192d45cba99625d022fb50d9/roles/580aa7dbb38d4cff8f9d1bb8983f99ef` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:55:27.525257 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b535bc7-ce22-4df5-987d-55fb7a64208b admin admin] RBAC: Authorizing `identity:create_grant(project_id=6383ffd740d947dba6cc24008d830ef5, user_id=51dce3e1192d45cba99625d022fb50d9, role_id=580aa7dbb38d4cff8f9d1bb8983f99ef)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:55:27.527900 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b535bc7-ce22-4df5-987d-55fb7a64208b admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:55:27.604836 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1390/2779] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Aug 24 06:55:27 2022] PUT /identity/v3/projects/6383ffd740d947dba6cc24008d830ef5/users/51dce3e1192d45cba99625d022fb50d9/roles/580aa7dbb38d4cff8f9d1bb8983f99ef => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:55:27.610144 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2960d180-6827-4391-a1ab-945e8a31b633 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:55:27.610437 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2960d180-6827-4391-a1ab-945e8a31b633 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:55:27.610717 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2960d180-6827-4391-a1ab-945e8a31b633 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:55:27.629956 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-2960d180-6827-4391-a1ab-945e8a31b633 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:55:27.631155 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-2960d180-6827-4391-a1ab-945e8a31b633 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:55:27.688877 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1390/2780] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:55:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:55:27.699156 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b535bc7-ce22-4df5-987d-55fb7a64208b admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:55:27.701004 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-5b535bc7-ce22-4df5-987d-55fb7a64208b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:55:27.766181 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b535bc7-ce22-4df5-987d-55fb7a64208b admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:55:27.769729 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e82aa2-c365-4161-9df3-c758e1ff93b8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bb6c79db75e4d169331c02133d1ec32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:55:27.770331 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e82aa2-c365-4161-9df3-c758e1ff93b8 service neutron] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:55:27.770610 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e82aa2-c365-4161-9df3-c758e1ff93b8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:55:27.770884 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e82aa2-c365-4161-9df3-c758e1ff93b8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:55:27.780320 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e82aa2-c365-4161-9df3-c758e1ff93b8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:55:27.781787 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e82aa2-c365-4161-9df3-c758e1ff93b8 service neutron] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:55:27.791042 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1391/2781] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:55:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:55:29.136245 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61ce756-1a96-4bd9-9aaa-1ee9ea845add None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:55:29.136785 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61ce756-1a96-4bd9-9aaa-1ee9ea845add None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:55:29.136874 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61ce756-1a96-4bd9-9aaa-1ee9ea845add None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:55:29.168508 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-a61ce756-1a96-4bd9-9aaa-1ee9ea845add None None] MFA Rules not processed for user `51dce3e1192d45cba99625d022fb50d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:55:29.171876 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-a61ce756-1a96-4bd9-9aaa-1ee9ea845add None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:55:29.244206 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1391/2782] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:55:29 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:55:29.248795 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e82aa2-c365-4161-9df3-c758e1ff93b8 service neutron] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:55:29.258470 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e82aa2-c365-4161-9df3-c758e1ff93b8 service neutron] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:55:29.263912 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84202e2-309f-426c-bd5b-8ef656f026c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:55:29.264593 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84202e2-309f-426c-bd5b-8ef656f026c3 admin admin] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:55:29.264877 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84202e2-309f-426c-bd5b-8ef656f026c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:55:29.265148 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84202e2-309f-426c-bd5b-8ef656f026c3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:55:29.265765 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84202e2-309f-426c-bd5b-8ef656f026c3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:55:29.267665 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84202e2-309f-426c-bd5b-8ef656f026c3 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:55:29.339832 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1392/2783] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Aug 24 06:55:29 2022] POST /identity/v3/projects => generated 373 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:55:29.344300 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61ce756-1a96-4bd9-9aaa-1ee9ea845add None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:55:29.354952 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61ce756-1a96-4bd9-9aaa-1ee9ea845add None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:55:29.371212 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:55:29.411938 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50a30e8-e717-464d-91db-a73090f8d1f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:55:29.412668 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50a30e8-e717-464d-91db-a73090f8d1f9 admin admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:55:29.412947 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50a30e8-e717-464d-91db-a73090f8d1f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:55:29.413244 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50a30e8-e717-464d-91db-a73090f8d1f9 admin admin] PATH_INFO: `/v3/users` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:55:29.413864 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50a30e8-e717-464d-91db-a73090f8d1f9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:55:29.415910 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50a30e8-e717-464d-91db-a73090f8d1f9 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:55:29.503280 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1392/2784] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Aug 24 06:55:29 2022] POST /identity/v3/users => generated 293 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:55:29.507365 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84202e2-309f-426c-bd5b-8ef656f026c3 admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:55:29.517824 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84202e2-309f-426c-bd5b-8ef656f026c3 admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:55:29.522437 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c2d055-17f1-4e66-9698-bd3def4f3f57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:55:29.523027 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c2d055-17f1-4e66-9698-bd3def4f3f57 admin admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:55:29.523302 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c2d055-17f1-4e66-9698-bd3def4f3f57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:55:29.523570 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c2d055-17f1-4e66-9698-bd3def4f3f57 admin admin] PATH_INFO: `/v3/roles` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:55:29.524193 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c2d055-17f1-4e66-9698-bd3def4f3f57 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:55:29.526468 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c2d055-17f1-4e66-9698-bd3def4f3f57 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:55:29.536869 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1393/2785] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Aug 24 06:55:29 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:55:29.541703 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50a30e8-e717-464d-91db-a73090f8d1f9 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:55:29.551697 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50a30e8-e717-464d-91db-a73090f8d1f9 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:55:29.556288 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509c3f44-0252-40b3-b57e-79ed17280926 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:55:29.556970 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509c3f44-0252-40b3-b57e-79ed17280926 admin admin] REQUEST_METHOD: `PUT` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:55:29.557253 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509c3f44-0252-40b3-b57e-79ed17280926 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:55:29.557524 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509c3f44-0252-40b3-b57e-79ed17280926 admin admin] PATH_INFO: `/v3/projects/9dee68f9f2c7460cb5203fca2737e608/users/2a52b6a126bb4d3e953d7f70a2e694b8/roles/49e47cd157d743f6905f1db07ec878a7` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:55:29.572352 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509c3f44-0252-40b3-b57e-79ed17280926 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9dee68f9f2c7460cb5203fca2737e608, user_id=2a52b6a126bb4d3e953d7f70a2e694b8, role_id=49e47cd157d743f6905f1db07ec878a7)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:55:29.575059 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509c3f44-0252-40b3-b57e-79ed17280926 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:55:29.656796 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1393/2786] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Aug 24 06:55:29 2022] PUT /identity/v3/projects/9dee68f9f2c7460cb5203fca2737e608/users/2a52b6a126bb4d3e953d7f70a2e694b8/roles/49e47cd157d743f6905f1db07ec878a7 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 06:55:29.660678 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c2d055-17f1-4e66-9698-bd3def4f3f57 admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:55:29.671648 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c2d055-17f1-4e66-9698-bd3def4f3f57 admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:55:29.685152 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=90,keystone:DELETE=6,keystone:INSERT=4 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:55:29.734853 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2918a582-2817-456a-abb0-fd8a77f2baa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:55:29.735497 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2918a582-2817-456a-abb0-fd8a77f2baa4 admin admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:55:29.735782 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2918a582-2817-456a-abb0-fd8a77f2baa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:55:29.736061 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2918a582-2817-456a-abb0-fd8a77f2baa4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:55:29.737436 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2918a582-2817-456a-abb0-fd8a77f2baa4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:55:29.738378 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2918a582-2817-456a-abb0-fd8a77f2baa4 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:55:29.749228 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1394/2787] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Aug 24 06:55:29 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:55:29.754896 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509c3f44-0252-40b3-b57e-79ed17280926 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:55:29.765359 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509c3f44-0252-40b3-b57e-79ed17280926 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:55:29.770164 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4c2212-2f93-48d3-903d-00d2c5aa4e81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:55:29.770810 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4c2212-2f93-48d3-903d-00d2c5aa4e81 admin admin] REQUEST_METHOD: `PUT` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:55:29.771089 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4c2212-2f93-48d3-903d-00d2c5aa4e81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:55:29.771362 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4c2212-2f93-48d3-903d-00d2c5aa4e81 admin admin] PATH_INFO: `/v3/projects/9dee68f9f2c7460cb5203fca2737e608/users/2a52b6a126bb4d3e953d7f70a2e694b8/roles/49e47cd157d743f6905f1db07ec878a7` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:55:29.773358 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4c2212-2f93-48d3-903d-00d2c5aa4e81 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9dee68f9f2c7460cb5203fca2737e608, user_id=2a52b6a126bb4d3e953d7f70a2e694b8, role_id=49e47cd157d743f6905f1db07ec878a7)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:55:29.776737 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4c2212-2f93-48d3-903d-00d2c5aa4e81 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:55:29.787305 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1394/2788] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Aug 24 06:55:29 2022] PUT /identity/v3/projects/9dee68f9f2c7460cb5203fca2737e608/users/2a52b6a126bb4d3e953d7f70a2e694b8/roles/49e47cd157d743f6905f1db07ec878a7 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:55:31.035894 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdc9bab-c2b3-4b4f-a81b-51dbee485c65 None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:55:31.036442 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdc9bab-c2b3-4b4f-a81b-51dbee485c65 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:55:31.036442 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdc9bab-c2b3-4b4f-a81b-51dbee485c65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:55:31.068540 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-cbdc9bab-c2b3-4b4f-a81b-51dbee485c65 None None] MFA Rules not processed for user `2a52b6a126bb4d3e953d7f70a2e694b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:55:31.069817 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-cbdc9bab-c2b3-4b4f-a81b-51dbee485c65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:55:31.142121 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1395/2789] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:55:31 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:55:31.154581 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4c2212-2f93-48d3-903d-00d2c5aa4e81 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:55:31.156307 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-ce4c2212-2f93-48d3-903d-00d2c5aa4e81 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:55:31.214211 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4c2212-2f93-48d3-903d-00d2c5aa4e81 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:55:31.217841 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8963974-c140-471d-a5a9-687db6308511 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c4def55ac0e48b29836bc5c64bc7402', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:55:31.218444 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8963974-c140-471d-a5a9-687db6308511 service cinder] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:55:31.218719 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8963974-c140-471d-a5a9-687db6308511 service cinder] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:55:31.218990 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8963974-c140-471d-a5a9-687db6308511 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:55:31.276486 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8963974-c140-471d-a5a9-687db6308511 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:55:31.277997 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8963974-c140-471d-a5a9-687db6308511 service cinder] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:55:31.287062 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1395/2790] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:55:31 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:55:33.051252 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdc9bab-c2b3-4b4f-a81b-51dbee485c65 None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:55:33.061773 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdc9bab-c2b3-4b4f-a81b-51dbee485c65 None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:55:33.065347 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ea881b-4740-4d46-a5bc-db93af1a2b09 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c4def55ac0e48b29836bc5c64bc7402', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:55:33.065926 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ea881b-4740-4d46-a5bc-db93af1a2b09 service cinder] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:55:33.066198 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ea881b-4740-4d46-a5bc-db93af1a2b09 service cinder] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:55:33.066471 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ea881b-4740-4d46-a5bc-db93af1a2b09 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:55:33.075814 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ea881b-4740-4d46-a5bc-db93af1a2b09 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:55:33.078636 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ea881b-4740-4d46-a5bc-db93af1a2b09 service cinder] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:55:33.086931 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1396/2791] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:55:33 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:55:41.282899 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=83,keystone:INSERT=5 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:55:43.082747 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=61 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:55:48.101975 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8963974-c140-471d-a5a9-687db6308511 service cinder] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:55:48.103873 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-d8963974-c140-471d-a5a9-687db6308511 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:55:48.174927 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8963974-c140-471d-a5a9-687db6308511 service cinder] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:55:48.179886 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc43198-24b5-4143-a721-ecc74670f7c1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abe9d8c9291845dfa4ef55034e5c9e32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:55:48.181151 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc43198-24b5-4143-a721-ecc74670f7c1 service placement] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:55:48.181433 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc43198-24b5-4143-a721-ecc74670f7c1 service placement] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:55:48.181754 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc43198-24b5-4143-a721-ecc74670f7c1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:55:48.183469 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-5fc43198-24b5-4143-a721-ecc74670f7c1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:55:48.195121 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc43198-24b5-4143-a721-ecc74670f7c1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:55:48.196809 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc43198-24b5-4143-a721-ecc74670f7c1 service placement] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:55:48.206635 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1396/2792] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:55:48 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:55:48.877927 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ea881b-4740-4d46-a5bc-db93af1a2b09 service cinder] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:55:48.942333 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ea881b-4740-4d46-a5bc-db93af1a2b09 service cinder] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:55:48.947282 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056a6666-2652-4a80-ac69-db362447e93d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:55:48.947911 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056a6666-2652-4a80-ac69-db362447e93d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:55:48.948189 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056a6666-2652-4a80-ac69-db362447e93d admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:55:48.948460 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056a6666-2652-4a80-ac69-db362447e93d admin admin] PATH_INFO: `/v3/users/51dce3e1192d45cba99625d022fb50d9` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:55:48.949619 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056a6666-2652-4a80-ac69-db362447e93d admin admin] RBAC: Authorizing `identity:delete_user(user_id=51dce3e1192d45cba99625d022fb50d9)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:55:48.951542 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056a6666-2652-4a80-ac69-db362447e93d admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:55:49.162644 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-056a6666-2652-4a80-ac69-db362447e93d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '51dce3e1192d45cba99625d022fb50d9'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:55:49.170146 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-056a6666-2652-4a80-ac69-db362447e93d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '51dce3e1192d45cba99625d022fb50d9'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:55:49.170971 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-056a6666-2652-4a80-ac69-db362447e93d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '51dce3e1192d45cba99625d022fb50d9'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:55:49.171598 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-056a6666-2652-4a80-ac69-db362447e93d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '51dce3e1192d45cba99625d022fb50d9'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:55:49.246538 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-056a6666-2652-4a80-ac69-db362447e93d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '51dce3e1192d45cba99625d022fb50d9'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:55:49.271773 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1397/2793] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Aug 24 06:55:48 2022] DELETE /identity/v3/users/51dce3e1192d45cba99625d022fb50d9 => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:55:49.513269 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc43198-24b5-4143-a721-ecc74670f7c1 service placement] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:55:49.524389 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc43198-24b5-4143-a721-ecc74670f7c1 service placement] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:55:49.582546 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0603b2c-d0b3-4a25-a9c1-940e0fb1b56c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:55:49.583181 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0603b2c-d0b3-4a25-a9c1-940e0fb1b56c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:55:49.583459 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0603b2c-d0b3-4a25-a9c1-940e0fb1b56c admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:55:49.583730 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0603b2c-d0b3-4a25-a9c1-940e0fb1b56c admin admin] PATH_INFO: `/v3/projects/6383ffd740d947dba6cc24008d830ef5` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:55:49.584889 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0603b2c-d0b3-4a25-a9c1-940e0fb1b56c admin admin] RBAC: Authorizing `identity:delete_project(project_id=6383ffd740d947dba6cc24008d830ef5)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:55:49.586901 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0603b2c-d0b3-4a25-a9c1-940e0fb1b56c admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:55:49.709368 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-f0603b2c-d0b3-4a25-a9c1-940e0fb1b56c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6383ffd740d947dba6cc24008d830ef5', 'request_id': 'req-f0603b2c-d0b3-4a25-a9c1-940e0fb1b56c'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:55:49.714983 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-f0603b2c-d0b3-4a25-a9c1-940e0fb1b56c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6383ffd740d947dba6cc24008d830ef5', 'request_id': 'req-f0603b2c-d0b3-4a25-a9c1-940e0fb1b56c'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:55:49.721441 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-f0603b2c-d0b3-4a25-a9c1-940e0fb1b56c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6383ffd740d947dba6cc24008d830ef5', 'request_id': 'req-f0603b2c-d0b3-4a25-a9c1-940e0fb1b56c'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:55:49.817891 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-f0603b2c-d0b3-4a25-a9c1-940e0fb1b56c admin admin] The token cache is being invalidate because project 6383ffd740d947dba6cc24008d830ef5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81852) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:55:49.818119 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-f0603b2c-d0b3-4a25-a9c1-940e0fb1b56c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:55:49.818753 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-f0603b2c-d0b3-4a25-a9c1-940e0fb1b56c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:55:49.820266 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1397/2794] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Aug 24 06:55:49 2022] DELETE /identity/v3/projects/6383ffd740d947dba6cc24008d830ef5 => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:55:49.824409 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056a6666-2652-4a80-ac69-db362447e93d admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:55:49.825890 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-056a6666-2652-4a80-ac69-db362447e93d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:55:49.839758 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056a6666-2652-4a80-ac69-db362447e93d admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:55:49.897034 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8fc732-33a0-48bf-82c8-6b4a83e95e31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:55:49.897748 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8fc732-33a0-48bf-82c8-6b4a83e95e31 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:55:49.897931 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8fc732-33a0-48bf-82c8-6b4a83e95e31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:55:49.898203 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8fc732-33a0-48bf-82c8-6b4a83e95e31 admin admin] PATH_INFO: `/v3/users/2a52b6a126bb4d3e953d7f70a2e694b8` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:55:49.899844 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8fc732-33a0-48bf-82c8-6b4a83e95e31 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2a52b6a126bb4d3e953d7f70a2e694b8)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:55:49.901835 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8fc732-33a0-48bf-82c8-6b4a83e95e31 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:55:49.924579 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:55:50.096775 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-3d8fc732-33a0-48bf-82c8-6b4a83e95e31 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2a52b6a126bb4d3e953d7f70a2e694b8'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:55:50.106444 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-3d8fc732-33a0-48bf-82c8-6b4a83e95e31 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2a52b6a126bb4d3e953d7f70a2e694b8'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:55:50.107564 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-3d8fc732-33a0-48bf-82c8-6b4a83e95e31 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2a52b6a126bb4d3e953d7f70a2e694b8'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:55:50.108606 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-3d8fc732-33a0-48bf-82c8-6b4a83e95e31 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2a52b6a126bb4d3e953d7f70a2e694b8'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:55:50.160609 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-3d8fc732-33a0-48bf-82c8-6b4a83e95e31 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2a52b6a126bb4d3e953d7f70a2e694b8'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:55:50.179217 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1398/2795] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Aug 24 06:55:49 2022] DELETE /identity/v3/users/2a52b6a126bb4d3e953d7f70a2e694b8 => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:55:50.419441 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0603b2c-d0b3-4a25-a9c1-940e0fb1b56c admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:55:50.423105 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-f0603b2c-d0b3-4a25-a9c1-940e0fb1b56c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:55:50.434696 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0603b2c-d0b3-4a25-a9c1-940e0fb1b56c admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:55:50.454056 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:55:50.495539 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24295093-4928-4a08-b692-9fce58721d19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:55:50.496351 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24295093-4928-4a08-b692-9fce58721d19 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:55:50.496542 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24295093-4928-4a08-b692-9fce58721d19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:55:50.497027 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24295093-4928-4a08-b692-9fce58721d19 admin admin] PATH_INFO: `/v3/projects/9dee68f9f2c7460cb5203fca2737e608` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:55:50.498769 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24295093-4928-4a08-b692-9fce58721d19 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9dee68f9f2c7460cb5203fca2737e608)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:55:50.500769 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24295093-4928-4a08-b692-9fce58721d19 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:55:50.618725 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-24295093-4928-4a08-b692-9fce58721d19 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9dee68f9f2c7460cb5203fca2737e608', 'request_id': 'req-24295093-4928-4a08-b692-9fce58721d19'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:55:50.624839 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-24295093-4928-4a08-b692-9fce58721d19 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9dee68f9f2c7460cb5203fca2737e608', 'request_id': 'req-24295093-4928-4a08-b692-9fce58721d19'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:55:50.631222 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-24295093-4928-4a08-b692-9fce58721d19 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9dee68f9f2c7460cb5203fca2737e608', 'request_id': 'req-24295093-4928-4a08-b692-9fce58721d19'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:55:50.731570 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-24295093-4928-4a08-b692-9fce58721d19 admin admin] The token cache is being invalidate because project 9dee68f9f2c7460cb5203fca2737e608 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81852) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:55:50.731883 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-24295093-4928-4a08-b692-9fce58721d19 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:55:50.732572 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-24295093-4928-4a08-b692-9fce58721d19 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:55:50.734091 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1398/2796] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Aug 24 06:55:50 2022] DELETE /identity/v3/projects/9dee68f9f2c7460cb5203fca2737e608 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:55:50.739085 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca3e14f-25f8-4edc-8cad-2e04352b0ade None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:55:50.743770 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca3e14f-25f8-4edc-8cad-2e04352b0ade None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:55:50.743770 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca3e14f-25f8-4edc-8cad-2e04352b0ade None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:55:50.761879 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-bca3e14f-25f8-4edc-8cad-2e04352b0ade None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:55:50.763083 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-bca3e14f-25f8-4edc-8cad-2e04352b0ade None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:55:50.823222 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1399/2797] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:55:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:55:50.831307 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dea3988-e9ed-4d72-8b9d-b6fc2e498e2d None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:55:50.831586 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dea3988-e9ed-4d72-8b9d-b6fc2e498e2d None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:55:50.831867 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dea3988-e9ed-4d72-8b9d-b6fc2e498e2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:55:50.852422 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-4dea3988-e9ed-4d72-8b9d-b6fc2e498e2d None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:55:50.853664 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-4dea3988-e9ed-4d72-8b9d-b6fc2e498e2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:55:50.861604 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1399/2798] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:55:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:55:50.866001 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca3e14f-25f8-4edc-8cad-2e04352b0ade None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:55:50.876581 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca3e14f-25f8-4edc-8cad-2e04352b0ade None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:55:50.881094 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809339e9-49cc-4a31-8c0a-21b01f1b4cd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:55:50.881701 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809339e9-49cc-4a31-8c0a-21b01f1b4cd9 admin admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:55:50.881979 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809339e9-49cc-4a31-8c0a-21b01f1b4cd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:55:50.882249 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809339e9-49cc-4a31-8c0a-21b01f1b4cd9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:55:50.882894 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809339e9-49cc-4a31-8c0a-21b01f1b4cd9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:55:50.883197 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809339e9-49cc-4a31-8c0a-21b01f1b4cd9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:55:50.885441 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809339e9-49cc-4a31-8c0a-21b01f1b4cd9 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:55:50.897412 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1400/2799] 10.222.0.62 () {58 vars in 1153 bytes} [Wed Aug 24 06:55:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:55:50.901771 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dea3988-e9ed-4d72-8b9d-b6fc2e498e2d None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:55:50.910902 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dea3988-e9ed-4d72-8b9d-b6fc2e498e2d None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:55:50.916063 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525133f4-2934-4686-9741-8a58b8a06b5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:55:50.916700 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525133f4-2934-4686-9741-8a58b8a06b5b admin admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:55:50.916984 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525133f4-2934-4686-9741-8a58b8a06b5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:55:50.917257 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525133f4-2934-4686-9741-8a58b8a06b5b admin admin] PATH_INFO: `/v3/projects` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:55:50.917939 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525133f4-2934-4686-9741-8a58b8a06b5b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:55:50.919810 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525133f4-2934-4686-9741-8a58b8a06b5b admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:55:51.023191 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1400/2800] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Aug 24 06:55:50 2022] POST /identity/v3/projects => generated 373 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:55:51.027342 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809339e9-49cc-4a31-8c0a-21b01f1b4cd9 admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:55:51.037901 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809339e9-49cc-4a31-8c0a-21b01f1b4cd9 admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:55:51.094332 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0c97a8-6f94-40b2-a5c5-ca97d0054d0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:55:51.094937 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0c97a8-6f94-40b2-a5c5-ca97d0054d0c admin admin] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:55:51.095220 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0c97a8-6f94-40b2-a5c5-ca97d0054d0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:55:51.095492 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0c97a8-6f94-40b2-a5c5-ca97d0054d0c admin admin] PATH_INFO: `/v3/users` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:55:51.096109 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0c97a8-6f94-40b2-a5c5-ca97d0054d0c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:55:51.098098 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0c97a8-6f94-40b2-a5c5-ca97d0054d0c admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:55:51.176335 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1401/2801] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Aug 24 06:55:51 2022] POST /identity/v3/users => generated 287 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:55:51.181299 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525133f4-2934-4686-9741-8a58b8a06b5b admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:55:51.191915 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525133f4-2934-4686-9741-8a58b8a06b5b admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:55:51.197820 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5876b2ae-ae8d-4db8-95f8-f4ca049440b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:55:51.198786 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5876b2ae-ae8d-4db8-95f8-f4ca049440b8 admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:55:51.198786 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5876b2ae-ae8d-4db8-95f8-f4ca049440b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:55:51.199148 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5876b2ae-ae8d-4db8-95f8-f4ca049440b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:55:51.199771 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5876b2ae-ae8d-4db8-95f8-f4ca049440b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:55:51.201274 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5876b2ae-ae8d-4db8-95f8-f4ca049440b8 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:55:51.212246 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1401/2802] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Aug 24 06:55:51 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:55:51.216545 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0c97a8-6f94-40b2-a5c5-ca97d0054d0c admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:55:51.226679 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0c97a8-6f94-40b2-a5c5-ca97d0054d0c admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:55:51.231860 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56431756-861b-49b8-a492-bde723812fd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:55:51.232461 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56431756-861b-49b8-a492-bde723812fd6 admin admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:55:51.232772 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56431756-861b-49b8-a492-bde723812fd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:55:51.233059 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56431756-861b-49b8-a492-bde723812fd6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:55:51.233716 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56431756-861b-49b8-a492-bde723812fd6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:55:51.235372 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56431756-861b-49b8-a492-bde723812fd6 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:55:51.246945 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1402/2803] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Aug 24 06:55:51 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:55:51.251381 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5876b2ae-ae8d-4db8-95f8-f4ca049440b8 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:55:51.262593 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5876b2ae-ae8d-4db8-95f8-f4ca049440b8 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:55:51.267494 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30cf64c-0153-4872-ac07-7f6ebde48e5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:55:51.268143 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30cf64c-0153-4872-ac07-7f6ebde48e5f admin admin] REQUEST_METHOD: `PUT` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:55:51.268418 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30cf64c-0153-4872-ac07-7f6ebde48e5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:55:51.268727 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30cf64c-0153-4872-ac07-7f6ebde48e5f admin admin] PATH_INFO: `/v3/projects/6e2becde3d02435ca94213e85878f2f2/users/a376ccc9ca874487bb7ec4be55b8f2e4/roles/580aa7dbb38d4cff8f9d1bb8983f99ef` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:55:51.283984 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30cf64c-0153-4872-ac07-7f6ebde48e5f admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e2becde3d02435ca94213e85878f2f2, user_id=a376ccc9ca874487bb7ec4be55b8f2e4, role_id=580aa7dbb38d4cff8f9d1bb8983f99ef)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:55:51.286581 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30cf64c-0153-4872-ac07-7f6ebde48e5f admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:55:51.359861 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1402/2804] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Aug 24 06:55:51 2022] PUT /identity/v3/projects/6e2becde3d02435ca94213e85878f2f2/users/a376ccc9ca874487bb7ec4be55b8f2e4/roles/580aa7dbb38d4cff8f9d1bb8983f99ef => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 06:55:51.373946 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5130f4d3-c480-4f2a-958f-19fc0dc97155 None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:55:51.374250 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5130f4d3-c480-4f2a-958f-19fc0dc97155 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:55:51.374527 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5130f4d3-c480-4f2a-958f-19fc0dc97155 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:55:51.382758 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:55:51.403318 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-5130f4d3-c480-4f2a-958f-19fc0dc97155 None None] MFA Rules not processed for user `a376ccc9ca874487bb7ec4be55b8f2e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:55:51.404732 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-5130f4d3-c480-4f2a-958f-19fc0dc97155 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:55:51.469573 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1403/2805] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:55:51 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:55:51.481234 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30cf64c-0153-4872-ac07-7f6ebde48e5f admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:55:51.483237 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-d30cf64c-0153-4872-ac07-7f6ebde48e5f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:55:51.535400 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:55:51.541195 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30cf64c-0153-4872-ac07-7f6ebde48e5f admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:55:51.544903 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306e3266-f6fc-48dd-9ab2-b9571eebeed1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c4def55ac0e48b29836bc5c64bc7402', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:55:51.545507 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306e3266-f6fc-48dd-9ab2-b9571eebeed1 service cinder] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:55:51.552661 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306e3266-f6fc-48dd-9ab2-b9571eebeed1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:55:51.552661 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306e3266-f6fc-48dd-9ab2-b9571eebeed1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:55:51.554689 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306e3266-f6fc-48dd-9ab2-b9571eebeed1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:55:51.556196 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306e3266-f6fc-48dd-9ab2-b9571eebeed1 service cinder] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:55:51.565429 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1403/2806] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:55:51 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:56:01.501431 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=30 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:56:01.629027 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=5 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:56:03.111745 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5130f4d3-c480-4f2a-958f-19fc0dc97155 None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:56:03.199834 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5130f4d3-c480-4f2a-958f-19fc0dc97155 None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:56:03.204648 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c813951-28ee-430f-a070-cebd16790cd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:56:03.205303 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c813951-28ee-430f-a070-cebd16790cd0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:56:03.205590 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c813951-28ee-430f-a070-cebd16790cd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:56:03.205874 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c813951-28ee-430f-a070-cebd16790cd0 admin admin] PATH_INFO: `/v3/users/a376ccc9ca874487bb7ec4be55b8f2e4` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:56:03.207630 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c813951-28ee-430f-a070-cebd16790cd0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a376ccc9ca874487bb7ec4be55b8f2e4)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:56:03.209194 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c813951-28ee-430f-a070-cebd16790cd0 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:56:03.418156 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-4c813951-28ee-430f-a070-cebd16790cd0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a376ccc9ca874487bb7ec4be55b8f2e4'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:56:03.425103 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-4c813951-28ee-430f-a070-cebd16790cd0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a376ccc9ca874487bb7ec4be55b8f2e4'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:56:03.425762 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-4c813951-28ee-430f-a070-cebd16790cd0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a376ccc9ca874487bb7ec4be55b8f2e4'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:56:03.426747 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-4c813951-28ee-430f-a070-cebd16790cd0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a376ccc9ca874487bb7ec4be55b8f2e4'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:56:03.491451 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-4c813951-28ee-430f-a070-cebd16790cd0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a376ccc9ca874487bb7ec4be55b8f2e4'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:56:03.510994 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1404/2807] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Aug 24 06:56:03 2022] DELETE /identity/v3/users/a376ccc9ca874487bb7ec4be55b8f2e4 => generated 0 bytes in 400 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:56:03.515671 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0399d741-21af-4c34-9c52-2812c6695466 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:56:03.516161 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0399d741-21af-4c34-9c52-2812c6695466 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:56:03.516280 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0399d741-21af-4c34-9c52-2812c6695466 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:56:03.535019 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-0399d741-21af-4c34-9c52-2812c6695466 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:56:03.536297 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-0399d741-21af-4c34-9c52-2812c6695466 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:56:03.595840 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1404/2808] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:56:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:56:03.605765 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c813951-28ee-430f-a070-cebd16790cd0 admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:56:03.607200 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-4c813951-28ee-430f-a070-cebd16790cd0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:56:03.676042 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c813951-28ee-430f-a070-cebd16790cd0 admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:56:03.679524 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b8d341-139c-45c5-8a3f-eda19f075c50 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bb6c79db75e4d169331c02133d1ec32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:56:03.680123 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b8d341-139c-45c5-8a3f-eda19f075c50 service neutron] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:56:03.680989 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b8d341-139c-45c5-8a3f-eda19f075c50 service neutron] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:56:03.681261 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b8d341-139c-45c5-8a3f-eda19f075c50 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:56:03.687270 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:56:03.691143 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b8d341-139c-45c5-8a3f-eda19f075c50 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:56:03.692857 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b8d341-139c-45c5-8a3f-eda19f075c50 service neutron] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:56:03.702508 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1405/2809] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:56:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:56:04.099205 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0399d741-21af-4c34-9c52-2812c6695466 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:56:04.100916 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-0399d741-21af-4c34-9c52-2812c6695466 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:56:04.114144 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0399d741-21af-4c34-9c52-2812c6695466 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:56:04.118841 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4dadf5b-98fa-4c3a-9e12-098bd0f5ff10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:56:04.119478 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dadf5b-98fa-4c3a-9e12-098bd0f5ff10 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:56:04.119756 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dadf5b-98fa-4c3a-9e12-098bd0f5ff10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:56:04.120030 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dadf5b-98fa-4c3a-9e12-098bd0f5ff10 admin admin] PATH_INFO: `/v3/projects/6e2becde3d02435ca94213e85878f2f2` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:56:04.121226 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4dadf5b-98fa-4c3a-9e12-098bd0f5ff10 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6e2becde3d02435ca94213e85878f2f2)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:56:04.123297 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4dadf5b-98fa-4c3a-9e12-098bd0f5ff10 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:56:04.290209 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-f4dadf5b-98fa-4c3a-9e12-098bd0f5ff10 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6e2becde3d02435ca94213e85878f2f2', 'request_id': 'req-f4dadf5b-98fa-4c3a-9e12-098bd0f5ff10'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:56:04.296611 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-f4dadf5b-98fa-4c3a-9e12-098bd0f5ff10 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6e2becde3d02435ca94213e85878f2f2', 'request_id': 'req-f4dadf5b-98fa-4c3a-9e12-098bd0f5ff10'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:56:04.303479 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-f4dadf5b-98fa-4c3a-9e12-098bd0f5ff10 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6e2becde3d02435ca94213e85878f2f2', 'request_id': 'req-f4dadf5b-98fa-4c3a-9e12-098bd0f5ff10'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:56:04.387988 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-f4dadf5b-98fa-4c3a-9e12-098bd0f5ff10 admin admin] The token cache is being invalidate because project 6e2becde3d02435ca94213e85878f2f2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81852) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:56:04.388292 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-f4dadf5b-98fa-4c3a-9e12-098bd0f5ff10 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:56:04.389040 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-f4dadf5b-98fa-4c3a-9e12-098bd0f5ff10 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:56:04.390632 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1405/2810] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Aug 24 06:56:04 2022] DELETE /identity/v3/projects/6e2becde3d02435ca94213e85878f2f2 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:56:04.395884 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd63c132-d5d1-4f35-b961-4688f91e1d8f None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:56:04.396376 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd63c132-d5d1-4f35-b961-4688f91e1d8f None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:56:04.396466 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd63c132-d5d1-4f35-b961-4688f91e1d8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:56:04.417356 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-fd63c132-d5d1-4f35-b961-4688f91e1d8f None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:56:04.418558 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-fd63c132-d5d1-4f35-b961-4688f91e1d8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:56:04.477817 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1406/2811] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:56:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:56:04.486985 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db883475-f8b0-4400-8c50-82da7d38a652 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:56:04.487270 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db883475-f8b0-4400-8c50-82da7d38a652 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:56:04.487540 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db883475-f8b0-4400-8c50-82da7d38a652 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:56:04.506782 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-db883475-f8b0-4400-8c50-82da7d38a652 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:56:04.507988 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-db883475-f8b0-4400-8c50-82da7d38a652 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:56:04.516394 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1406/2812] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:56:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:56:04.520669 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd63c132-d5d1-4f35-b961-4688f91e1d8f None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:56:04.531116 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd63c132-d5d1-4f35-b961-4688f91e1d8f None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:56:04.535923 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53995f1-af1e-4afa-982e-f9926a9b9dab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:56:04.536542 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53995f1-af1e-4afa-982e-f9926a9b9dab admin admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:56:04.536829 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53995f1-af1e-4afa-982e-f9926a9b9dab admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:56:04.537105 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53995f1-af1e-4afa-982e-f9926a9b9dab admin admin] PATH_INFO: `/v3/domains` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:56:04.537782 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53995f1-af1e-4afa-982e-f9926a9b9dab admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:56:04.538085 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53995f1-af1e-4afa-982e-f9926a9b9dab admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:56:04.539738 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53995f1-af1e-4afa-982e-f9926a9b9dab admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:56:04.551470 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1407/2813] 10.222.0.62 () {58 vars in 1153 bytes} [Wed Aug 24 06:56:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:56:04.555779 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db883475-f8b0-4400-8c50-82da7d38a652 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:56:04.565141 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db883475-f8b0-4400-8c50-82da7d38a652 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:56:04.570609 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032b62e1-3c10-4e6b-bb7e-98a226da6d15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:56:04.571730 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032b62e1-3c10-4e6b-bb7e-98a226da6d15 admin admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:56:04.572005 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032b62e1-3c10-4e6b-bb7e-98a226da6d15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:56:04.572271 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032b62e1-3c10-4e6b-bb7e-98a226da6d15 admin admin] PATH_INFO: `/v3/projects` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:56:04.572930 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032b62e1-3c10-4e6b-bb7e-98a226da6d15 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:56:04.574877 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032b62e1-3c10-4e6b-bb7e-98a226da6d15 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:56:04.674342 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1407/2814] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Aug 24 06:56:04 2022] POST /identity/v3/projects => generated 379 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:56:04.678351 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53995f1-af1e-4afa-982e-f9926a9b9dab admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:56:04.688622 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53995f1-af1e-4afa-982e-f9926a9b9dab admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:56:04.746676 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949b3532-3434-42a9-81cc-7fe59aaeaa99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:56:04.747282 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949b3532-3434-42a9-81cc-7fe59aaeaa99 admin admin] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:56:04.747557 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949b3532-3434-42a9-81cc-7fe59aaeaa99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:56:04.747825 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949b3532-3434-42a9-81cc-7fe59aaeaa99 admin admin] PATH_INFO: `/v3/users` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:56:04.748448 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949b3532-3434-42a9-81cc-7fe59aaeaa99 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:56:04.750398 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949b3532-3434-42a9-81cc-7fe59aaeaa99 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:56:04.857777 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1408/2815] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Aug 24 06:56:04 2022] POST /identity/v3/users => generated 290 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 06:56:04.861996 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032b62e1-3c10-4e6b-bb7e-98a226da6d15 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:56:04.872759 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032b62e1-3c10-4e6b-bb7e-98a226da6d15 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:56:04.877445 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f42414-c3be-48a3-9067-259211dffc22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:56:04.878039 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f42414-c3be-48a3-9067-259211dffc22 admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:56:04.878312 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f42414-c3be-48a3-9067-259211dffc22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:56:04.878579 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f42414-c3be-48a3-9067-259211dffc22 admin admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:56:04.879206 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f42414-c3be-48a3-9067-259211dffc22 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:56:04.880817 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f42414-c3be-48a3-9067-259211dffc22 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:56:04.892197 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1408/2816] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Aug 24 06:56:04 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:56:04.896205 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949b3532-3434-42a9-81cc-7fe59aaeaa99 admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:56:04.906096 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949b3532-3434-42a9-81cc-7fe59aaeaa99 admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:56:04.910708 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942cbe96-4496-424d-8f23-ff04242253f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:56:04.911309 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942cbe96-4496-424d-8f23-ff04242253f7 admin admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:56:04.911616 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942cbe96-4496-424d-8f23-ff04242253f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:56:04.911890 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942cbe96-4496-424d-8f23-ff04242253f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:56:04.912556 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942cbe96-4496-424d-8f23-ff04242253f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:56:04.914135 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942cbe96-4496-424d-8f23-ff04242253f7 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:56:04.925681 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1409/2817] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Aug 24 06:56:04 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:56:04.929916 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f42414-c3be-48a3-9067-259211dffc22 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:56:04.940462 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f42414-c3be-48a3-9067-259211dffc22 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:56:04.945367 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e99b43-af07-4659-ab64-e8646c6403c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:56:04.946672 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e99b43-af07-4659-ab64-e8646c6403c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:56:04.946951 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e99b43-af07-4659-ab64-e8646c6403c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:56:04.947225 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e99b43-af07-4659-ab64-e8646c6403c9 admin admin] PATH_INFO: `/v3/projects/01a59a0e0c9e4e8fa1c0e147e720eccb/users/86ff51618d4b4b71bfad73467ddc8585/roles/580aa7dbb38d4cff8f9d1bb8983f99ef` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:56:04.961534 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e99b43-af07-4659-ab64-e8646c6403c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=01a59a0e0c9e4e8fa1c0e147e720eccb, user_id=86ff51618d4b4b71bfad73467ddc8585, role_id=580aa7dbb38d4cff8f9d1bb8983f99ef)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:56:04.964174 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e99b43-af07-4659-ab64-e8646c6403c9 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:56:05.126516 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1409/2818] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Aug 24 06:56:04 2022] PUT /identity/v3/projects/01a59a0e0c9e4e8fa1c0e147e720eccb/users/86ff51618d4b4b71bfad73467ddc8585/roles/580aa7dbb38d4cff8f9d1bb8983f99ef => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:56:05.131682 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61ebef8-c8e7-4d74-b17e-69accfe8d633 None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:56:05.131963 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61ebef8-c8e7-4d74-b17e-69accfe8d633 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:56:05.132236 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61ebef8-c8e7-4d74-b17e-69accfe8d633 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:56:05.152047 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-e61ebef8-c8e7-4d74-b17e-69accfe8d633 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:56:05.153338 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-e61ebef8-c8e7-4d74-b17e-69accfe8d633 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:56:05.203950 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:56:05.212126 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1410/2819] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:56:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:56:05.221878 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e99b43-af07-4659-ab64-e8646c6403c9 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:56:05.223593 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-f0e99b43-af07-4659-ab64-e8646c6403c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:56:05.246266 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:56:05.280955 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e99b43-af07-4659-ab64-e8646c6403c9 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:56:05.284549 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa53d58-78e7-48b5-993f-291224d41b83 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bb6c79db75e4d169331c02133d1ec32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:56:05.285149 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa53d58-78e7-48b5-993f-291224d41b83 service neutron] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:56:05.285424 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa53d58-78e7-48b5-993f-291224d41b83 service neutron] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:56:05.285694 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa53d58-78e7-48b5-993f-291224d41b83 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:56:05.295008 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa53d58-78e7-48b5-993f-291224d41b83 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:56:05.296549 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa53d58-78e7-48b5-993f-291224d41b83 service neutron] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:56:05.307333 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1410/2820] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:56:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:56:08.871105 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61ebef8-c8e7-4d74-b17e-69accfe8d633 None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:56:08.873408 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-e61ebef8-c8e7-4d74-b17e-69accfe8d633 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:56:08.988288 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61ebef8-c8e7-4d74-b17e-69accfe8d633 None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:56:08.995637 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866fadd8-4677-4721-be5a-c20fa83368ae service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abe9d8c9291845dfa4ef55034e5c9e32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:56:08.996273 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866fadd8-4677-4721-be5a-c20fa83368ae service placement] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:56:08.997028 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866fadd8-4677-4721-be5a-c20fa83368ae service placement] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:56:08.997028 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866fadd8-4677-4721-be5a-c20fa83368ae service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:56:08.998668 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-866fadd8-4677-4721-be5a-c20fa83368ae service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:56:09.012302 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866fadd8-4677-4721-be5a-c20fa83368ae service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:56:09.014014 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866fadd8-4677-4721-be5a-c20fa83368ae service placement] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:56:09.025662 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1411/2821] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:56:08 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:56:10.959792 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9125e4f5-d9d0-423a-b7cb-3ec11f8b7211 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:56:10.960278 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9125e4f5-d9d0-423a-b7cb-3ec11f8b7211 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:56:10.960403 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9125e4f5-d9d0-423a-b7cb-3ec11f8b7211 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:56:10.989183 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-9125e4f5-d9d0-423a-b7cb-3ec11f8b7211 None None] MFA Rules not processed for user `86ff51618d4b4b71bfad73467ddc8585`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:56:10.990426 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-9125e4f5-d9d0-423a-b7cb-3ec11f8b7211 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:56:11.055805 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1411/2822] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:56:10 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:56:11.060678 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866fadd8-4677-4721-be5a-c20fa83368ae service placement] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:56:11.071868 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866fadd8-4677-4721-be5a-c20fa83368ae service placement] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:56:11.076524 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dda561d-07c8-4fd3-8788-57d7cb8e4964 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:56:11.077021 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dda561d-07c8-4fd3-8788-57d7cb8e4964 admin admin] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:56:11.077298 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dda561d-07c8-4fd3-8788-57d7cb8e4964 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:56:11.077568 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dda561d-07c8-4fd3-8788-57d7cb8e4964 admin admin] PATH_INFO: `/v3/projects` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:56:11.078181 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dda561d-07c8-4fd3-8788-57d7cb8e4964 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:56:11.080079 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dda561d-07c8-4fd3-8788-57d7cb8e4964 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:56:11.178515 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1412/2823] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Aug 24 06:56:11 2022] POST /identity/v3/projects => generated 377 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 06:56:11.183293 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9125e4f5-d9d0-423a-b7cb-3ec11f8b7211 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:56:11.196683 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9125e4f5-d9d0-423a-b7cb-3ec11f8b7211 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:56:11.264832 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42bd439-8542-43b1-9382-bda8d4dc95b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:56:11.265468 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42bd439-8542-43b1-9382-bda8d4dc95b2 admin admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:56:11.265743 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42bd439-8542-43b1-9382-bda8d4dc95b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:56:11.266009 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42bd439-8542-43b1-9382-bda8d4dc95b2 admin admin] PATH_INFO: `/v3/users` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:56:11.266642 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42bd439-8542-43b1-9382-bda8d4dc95b2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:56:11.268653 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42bd439-8542-43b1-9382-bda8d4dc95b2 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:56:11.376735 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1412/2824] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Aug 24 06:56:11 2022] POST /identity/v3/users => generated 295 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:56:11.380888 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dda561d-07c8-4fd3-8788-57d7cb8e4964 admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:56:11.391799 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dda561d-07c8-4fd3-8788-57d7cb8e4964 admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:56:11.396745 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459d4ca8-b3f4-4fc1-8fd9-a858aa177293 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:56:11.397374 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459d4ca8-b3f4-4fc1-8fd9-a858aa177293 admin admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:56:11.397674 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459d4ca8-b3f4-4fc1-8fd9-a858aa177293 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:56:11.397968 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459d4ca8-b3f4-4fc1-8fd9-a858aa177293 admin admin] PATH_INFO: `/v3/roles` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:56:11.398617 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459d4ca8-b3f4-4fc1-8fd9-a858aa177293 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:56:11.400292 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459d4ca8-b3f4-4fc1-8fd9-a858aa177293 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:56:11.410933 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1413/2825] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Aug 24 06:56:11 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 06:56:11.415920 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42bd439-8542-43b1-9382-bda8d4dc95b2 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:56:11.425994 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42bd439-8542-43b1-9382-bda8d4dc95b2 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:56:11.431433 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f00ed7-8a01-4231-b653-46df53c0116f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:56:11.432108 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f00ed7-8a01-4231-b653-46df53c0116f admin admin] REQUEST_METHOD: `PUT` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:56:11.432397 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f00ed7-8a01-4231-b653-46df53c0116f admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:56:11.433272 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f00ed7-8a01-4231-b653-46df53c0116f admin admin] PATH_INFO: `/v3/projects/17dc8467da014d3c83a2c2de396ea834/users/6c1ce827952f4339bf117b9e7cd431d9/roles/49e47cd157d743f6905f1db07ec878a7` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:56:11.449999 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f00ed7-8a01-4231-b653-46df53c0116f admin admin] RBAC: Authorizing `identity:create_grant(project_id=17dc8467da014d3c83a2c2de396ea834, user_id=6c1ce827952f4339bf117b9e7cd431d9, role_id=49e47cd157d743f6905f1db07ec878a7)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:56:11.452710 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f00ed7-8a01-4231-b653-46df53c0116f admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:56:11.533467 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1413/2826] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Aug 24 06:56:11 2022] PUT /identity/v3/projects/17dc8467da014d3c83a2c2de396ea834/users/6c1ce827952f4339bf117b9e7cd431d9/roles/49e47cd157d743f6905f1db07ec878a7 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:56:11.537506 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459d4ca8-b3f4-4fc1-8fd9-a858aa177293 admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:56:11.548306 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459d4ca8-b3f4-4fc1-8fd9-a858aa177293 admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:56:11.606554 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0754b136-4f73-4710-8ba0-58efcd98f2de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:56:11.607083 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0754b136-4f73-4710-8ba0-58efcd98f2de admin admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:56:11.607359 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0754b136-4f73-4710-8ba0-58efcd98f2de admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:56:11.607628 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0754b136-4f73-4710-8ba0-58efcd98f2de admin admin] PATH_INFO: `/v3/roles` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:56:11.608264 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0754b136-4f73-4710-8ba0-58efcd98f2de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:56:11.609903 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0754b136-4f73-4710-8ba0-58efcd98f2de admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:56:11.619660 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1414/2827] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Aug 24 06:56:11 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:56:11.624061 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f00ed7-8a01-4231-b653-46df53c0116f admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:56:11.635972 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f00ed7-8a01-4231-b653-46df53c0116f admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:56:11.641840 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6777e4f2-f7fb-4546-9219-b95dd0106f84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:56:11.642561 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6777e4f2-f7fb-4546-9219-b95dd0106f84 admin admin] REQUEST_METHOD: `PUT` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:56:11.642843 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6777e4f2-f7fb-4546-9219-b95dd0106f84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:56:11.643116 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6777e4f2-f7fb-4546-9219-b95dd0106f84 admin admin] PATH_INFO: `/v3/projects/17dc8467da014d3c83a2c2de396ea834/users/6c1ce827952f4339bf117b9e7cd431d9/roles/49e47cd157d743f6905f1db07ec878a7` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:56:11.645091 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6777e4f2-f7fb-4546-9219-b95dd0106f84 admin admin] RBAC: Authorizing `identity:create_grant(project_id=17dc8467da014d3c83a2c2de396ea834, user_id=6c1ce827952f4339bf117b9e7cd431d9, role_id=49e47cd157d743f6905f1db07ec878a7)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:56:11.647828 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6777e4f2-f7fb-4546-9219-b95dd0106f84 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:56:11.660094 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1414/2828] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Aug 24 06:56:11 2022] PUT /identity/v3/projects/17dc8467da014d3c83a2c2de396ea834/users/6c1ce827952f4339bf117b9e7cd431d9/roles/49e47cd157d743f6905f1db07ec878a7 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:56:17.416487 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eee7249-e5c7-4c0b-b56c-ff6b735c874c None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:56:17.420829 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eee7249-e5c7-4c0b-b56c-ff6b735c874c None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:56:17.420829 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eee7249-e5c7-4c0b-b56c-ff6b735c874c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:56:17.446379 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-4eee7249-e5c7-4c0b-b56c-ff6b735c874c None None] MFA Rules not processed for user `6c1ce827952f4339bf117b9e7cd431d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:56:17.447525 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-4eee7249-e5c7-4c0b-b56c-ff6b735c874c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:56:17.469557 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:56:17.519318 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1415/2829] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:56:17 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:56:17.535244 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6777e4f2-f7fb-4546-9219-b95dd0106f84 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:56:17.537167 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-6777e4f2-f7fb-4546-9219-b95dd0106f84 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:56:17.544171 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:56:17.610476 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6777e4f2-f7fb-4546-9219-b95dd0106f84 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:56:17.616294 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7b1409-040b-4340-92c0-434dc1bd3fbe service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb6d3743340241f98cacb9e0d063298c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:56:17.616961 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7b1409-040b-4340-92c0-434dc1bd3fbe service nova] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:56:17.617267 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7b1409-040b-4340-92c0-434dc1bd3fbe service nova] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:56:17.618058 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7b1409-040b-4340-92c0-434dc1bd3fbe service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:56:17.677905 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7b1409-040b-4340-92c0-434dc1bd3fbe service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:56:17.679695 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7b1409-040b-4340-92c0-434dc1bd3fbe service nova] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:56:17.690378 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1415/2830] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:56:17 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:56:27.680638 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=19 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:56:27.709335 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=54 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:56:36.036127 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8385c99f-d59f-4422-8a56-fb0946665a5c None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:56:36.036707 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8385c99f-d59f-4422-8a56-fb0946665a5c None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:56:36.036707 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8385c99f-d59f-4422-8a56-fb0946665a5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:56:36.059099 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-8385c99f-d59f-4422-8a56-fb0946665a5c None None] MFA Rules not processed for user `86ff51618d4b4b71bfad73467ddc8585`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:56:36.060351 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-8385c99f-d59f-4422-8a56-fb0946665a5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:56:36.067503 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1416/2831] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:56:36 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:56:36.382776 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7b1409-040b-4340-92c0-434dc1bd3fbe service nova] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:56:36.439544 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7b1409-040b-4340-92c0-434dc1bd3fbe service nova] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:56:36.443117 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb936f49-f68c-4120-956e-6f8c03389552 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bb6c79db75e4d169331c02133d1ec32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:56:36.443971 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb936f49-f68c-4120-956e-6f8c03389552 service neutron] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:56:36.444242 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb936f49-f68c-4120-956e-6f8c03389552 service neutron] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:56:36.444527 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb936f49-f68c-4120-956e-6f8c03389552 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:56:36.446245 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-cb936f49-f68c-4120-956e-6f8c03389552 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:56:36.455703 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb936f49-f68c-4120-956e-6f8c03389552 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:56:36.457155 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb936f49-f68c-4120-956e-6f8c03389552 service neutron] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:56:36.465568 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1416/2832] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:56:36 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:56:38.138778 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8385c99f-d59f-4422-8a56-fb0946665a5c None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:56:38.140244 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-8385c99f-d59f-4422-8a56-fb0946665a5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:56:38.214235 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8385c99f-d59f-4422-8a56-fb0946665a5c None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:56:38.219334 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e2ab8b-714a-4045-aa08-d3e2c6ee9cc9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abe9d8c9291845dfa4ef55034e5c9e32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:56:38.220015 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e2ab8b-714a-4045-aa08-d3e2c6ee9cc9 service placement] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:56:38.220298 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e2ab8b-714a-4045-aa08-d3e2c6ee9cc9 service placement] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:56:38.220605 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e2ab8b-714a-4045-aa08-d3e2c6ee9cc9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:56:38.222383 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-52e2ab8b-714a-4045-aa08-d3e2c6ee9cc9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:56:38.234248 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e2ab8b-714a-4045-aa08-d3e2c6ee9cc9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:56:38.235821 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e2ab8b-714a-4045-aa08-d3e2c6ee9cc9 service placement] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:56:38.246274 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1417/2833] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:56:38 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:56:41.387932 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb936f49-f68c-4120-956e-6f8c03389552 service neutron] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:56:41.389443 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-cb936f49-f68c-4120-956e-6f8c03389552 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:56:41.400567 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb936f49-f68c-4120-956e-6f8c03389552 service neutron] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:56:41.410538 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f4203f-e2bc-4fc6-a196-66327f2baab0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bb6c79db75e4d169331c02133d1ec32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:56:41.410538 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f4203f-e2bc-4fc6-a196-66327f2baab0 service neutron] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:56:41.410538 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f4203f-e2bc-4fc6-a196-66327f2baab0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:56:41.410538 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f4203f-e2bc-4fc6-a196-66327f2baab0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:56:41.410917 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-36f4203f-e2bc-4fc6-a196-66327f2baab0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:56:41.420797 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f4203f-e2bc-4fc6-a196-66327f2baab0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:56:41.422925 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f4203f-e2bc-4fc6-a196-66327f2baab0 service neutron] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:56:41.434117 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1417/2834] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:56:41 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:56:42.264861 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e2ab8b-714a-4045-aa08-d3e2c6ee9cc9 service placement] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:56:42.277772 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e2ab8b-714a-4045-aa08-d3e2c6ee9cc9 service placement] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:56:42.283372 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331e9171-1eb7-4661-a794-4a3d12464dcb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb6d3743340241f98cacb9e0d063298c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:56:42.283977 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331e9171-1eb7-4661-a794-4a3d12464dcb service nova] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:56:42.284256 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331e9171-1eb7-4661-a794-4a3d12464dcb service nova] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:56:42.284559 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331e9171-1eb7-4661-a794-4a3d12464dcb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:56:42.286298 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-331e9171-1eb7-4661-a794-4a3d12464dcb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:56:42.300086 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331e9171-1eb7-4661-a794-4a3d12464dcb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:56:42.301711 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331e9171-1eb7-4661-a794-4a3d12464dcb service nova] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:56:42.314614 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1418/2835] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:56:42 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:56:51.427902 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=32 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:56:52.308789 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=41 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:56:53.214826 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f4203f-e2bc-4fc6-a196-66327f2baab0 service neutron] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:56:53.256973 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f4203f-e2bc-4fc6-a196-66327f2baab0 service neutron] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:56:53.262906 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7b9c2f-e895-4391-a595-98a7b3c41059 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb6d3743340241f98cacb9e0d063298c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:56:53.263580 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7b9c2f-e895-4391-a595-98a7b3c41059 service nova] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:56:53.263860 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7b9c2f-e895-4391-a595-98a7b3c41059 service nova] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:56:53.264133 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7b9c2f-e895-4391-a595-98a7b3c41059 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:56:53.265922 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-fd7b9c2f-e895-4391-a595-98a7b3c41059 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:56:53.304266 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7b9c2f-e895-4391-a595-98a7b3c41059 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:56:53.307325 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7b9c2f-e895-4391-a595-98a7b3c41059 service nova] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:56:53.320210 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1418/2836] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:56:53 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:56:55.865741 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331e9171-1eb7-4661-a794-4a3d12464dcb service nova] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:56:55.895229 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331e9171-1eb7-4661-a794-4a3d12464dcb service nova] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:56:55.899511 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e6f786-32e3-4f39-be0e-da936f0de7b9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bb6c79db75e4d169331c02133d1ec32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:56:55.900117 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e6f786-32e3-4f39-be0e-da936f0de7b9 service neutron] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:56:55.900398 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e6f786-32e3-4f39-be0e-da936f0de7b9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:56:55.900698 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e6f786-32e3-4f39-be0e-da936f0de7b9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:56:55.909528 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e6f786-32e3-4f39-be0e-da936f0de7b9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:56:55.910936 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e6f786-32e3-4f39-be0e-da936f0de7b9 service neutron] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:56:55.920670 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1419/2837] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:56:55 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 06:56:58.130183 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7b9c2f-e895-4391-a595-98a7b3c41059 service nova] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:56:58.142672 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7b9c2f-e895-4391-a595-98a7b3c41059 service nova] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:56:58.148416 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f2120e-97b8-4073-9c48-9dd74bcff63e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb6d3743340241f98cacb9e0d063298c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:56:58.149197 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f2120e-97b8-4073-9c48-9dd74bcff63e service nova] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:56:58.149540 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f2120e-97b8-4073-9c48-9dd74bcff63e service nova] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:56:58.149881 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f2120e-97b8-4073-9c48-9dd74bcff63e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:56:58.151794 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-c3f2120e-97b8-4073-9c48-9dd74bcff63e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:56:58.164591 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f2120e-97b8-4073-9c48-9dd74bcff63e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:56:58.166442 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f2120e-97b8-4073-9c48-9dd74bcff63e service nova] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:56:58.177660 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1419/2838] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:56:58 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:57:05.916671 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=15 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:57:08.172968 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=33 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:58:37.272718 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e6f786-32e3-4f39-be0e-da936f0de7b9 service neutron] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:58:37.274520 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-14e6f786-32e3-4f39-be0e-da936f0de7b9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:58:37.345290 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e6f786-32e3-4f39-be0e-da936f0de7b9 service neutron] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:58:37.349080 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f3f672-9a0a-450e-9c6c-3cafd6aaeb1c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c4def55ac0e48b29836bc5c64bc7402', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:58:37.350358 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f3f672-9a0a-450e-9c6c-3cafd6aaeb1c service cinder] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:58:37.350623 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f3f672-9a0a-450e-9c6c-3cafd6aaeb1c service cinder] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:58:37.350896 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f3f672-9a0a-450e-9c6c-3cafd6aaeb1c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:58:37.352828 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-f3f3f672-9a0a-450e-9c6c-3cafd6aaeb1c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:58:37.427112 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f3f672-9a0a-450e-9c6c-3cafd6aaeb1c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:58:37.429974 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f3f672-9a0a-450e-9c6c-3cafd6aaeb1c service cinder] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:58:37.447384 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1420/2839] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:58:37 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:58:47.439194 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=48 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:58:58.628415 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babc4c4d-4389-4cc2-a69b-68270371438e None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:58:58.629264 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babc4c4d-4389-4cc2-a69b-68270371438e None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:58:58.629264 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babc4c4d-4389-4cc2-a69b-68270371438e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:58:58.648845 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-babc4c4d-4389-4cc2-a69b-68270371438e None None] MFA Rules not processed for user `86ff51618d4b4b71bfad73467ddc8585`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:58:58.650110 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-babc4c4d-4389-4cc2-a69b-68270371438e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:58:58.657589 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1420/2840] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 06:58:58 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:58:59.026357 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f3f672-9a0a-450e-9c6c-3cafd6aaeb1c service cinder] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:58:59.036402 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f3f672-9a0a-450e-9c6c-3cafd6aaeb1c service cinder] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:58:59.040964 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2799db58-824b-4c73-94c1-b0487d7b4858 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bb6c79db75e4d169331c02133d1ec32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:58:59.041588 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2799db58-824b-4c73-94c1-b0487d7b4858 service neutron] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:58:59.041863 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2799db58-824b-4c73-94c1-b0487d7b4858 service neutron] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:58:59.042132 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2799db58-824b-4c73-94c1-b0487d7b4858 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:58:59.044015 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-2799db58-824b-4c73-94c1-b0487d7b4858 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:58:59.076314 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2799db58-824b-4c73-94c1-b0487d7b4858 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:58:59.077814 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2799db58-824b-4c73-94c1-b0487d7b4858 service neutron] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:58:59.088809 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1421/2841] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 06:58:59 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:59:08.643278 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=3 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:59:09.083369 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=11 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:01:01.088226 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3764118f-d59b-45c3-8eca-63b56d1e450a None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:01:01.088848 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3764118f-d59b-45c3-8eca-63b56d1e450a None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:01:01.088848 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3764118f-d59b-45c3-8eca-63b56d1e450a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:01:01.108383 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-3764118f-d59b-45c3-8eca-63b56d1e450a None None] MFA Rules not processed for user `86ff51618d4b4b71bfad73467ddc8585`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:01:01.109685 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-3764118f-d59b-45c3-8eca-63b56d1e450a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:01:01.116825 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1421/2842] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 07:01:01 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:01:01.158961 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2799db58-824b-4c73-94c1-b0487d7b4858 service neutron] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:01:01.180222 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2799db58-824b-4c73-94c1-b0487d7b4858 service neutron] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:01:01.184107 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b81dbd-3c46-4f59-8c2c-6dc3e51527f6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c4def55ac0e48b29836bc5c64bc7402', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:01:01.184872 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b81dbd-3c46-4f59-8c2c-6dc3e51527f6 service cinder] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:01:01.185153 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b81dbd-3c46-4f59-8c2c-6dc3e51527f6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:01:01.185426 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b81dbd-3c46-4f59-8c2c-6dc3e51527f6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:01:01.187249 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-a0b81dbd-3c46-4f59-8c2c-6dc3e51527f6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:01:01.202988 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b81dbd-3c46-4f59-8c2c-6dc3e51527f6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:01:01.204514 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b81dbd-3c46-4f59-8c2c-6dc3e51527f6 service cinder] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:01:01.217799 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1422/2843] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 07:01:01 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:01:02.630945 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3764118f-d59b-45c3-8eca-63b56d1e450a None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:01:02.646685 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3764118f-d59b-45c3-8eca-63b56d1e450a None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:01:02.650451 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872197d6-707b-4c41-a6eb-c8d39d00382e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bb6c79db75e4d169331c02133d1ec32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:01:02.651072 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872197d6-707b-4c41-a6eb-c8d39d00382e service neutron] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:01:02.651349 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872197d6-707b-4c41-a6eb-c8d39d00382e service neutron] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:01:02.651620 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872197d6-707b-4c41-a6eb-c8d39d00382e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:01:02.654123 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-872197d6-707b-4c41-a6eb-c8d39d00382e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:01:02.669038 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872197d6-707b-4c41-a6eb-c8d39d00382e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:01:02.670500 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872197d6-707b-4c41-a6eb-c8d39d00382e service neutron] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:01:02.692838 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1422/2844] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 07:01:02 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 07:01:11.212051 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=10 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:01:12.680286 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=9 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:01:14.882151 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b81dbd-3c46-4f59-8c2c-6dc3e51527f6 service cinder] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:01:14.896740 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b81dbd-3c46-4f59-8c2c-6dc3e51527f6 service cinder] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:01:14.902130 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3905ce6-e3ad-4a88-9d19-9501fed40e6d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abe9d8c9291845dfa4ef55034e5c9e32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:01:14.902739 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3905ce6-e3ad-4a88-9d19-9501fed40e6d service placement] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:01:14.903024 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3905ce6-e3ad-4a88-9d19-9501fed40e6d service placement] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:01:14.903303 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3905ce6-e3ad-4a88-9d19-9501fed40e6d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:01:14.914165 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3905ce6-e3ad-4a88-9d19-9501fed40e6d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:01:14.915777 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3905ce6-e3ad-4a88-9d19-9501fed40e6d service placement] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:01:14.927198 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1423/2845] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 07:01:14 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:01:18.157018 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872197d6-707b-4c41-a6eb-c8d39d00382e service neutron] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:01:18.158527 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-872197d6-707b-4c41-a6eb-c8d39d00382e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:01:18.172614 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872197d6-707b-4c41-a6eb-c8d39d00382e service neutron] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:01:18.178784 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1090be-77d8-4c8c-9d19-492da92ef24c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb6d3743340241f98cacb9e0d063298c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:01:18.179400 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1090be-77d8-4c8c-9d19-492da92ef24c service nova] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:01:18.179679 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1090be-77d8-4c8c-9d19-492da92ef24c service nova] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:01:18.179954 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1090be-77d8-4c8c-9d19-492da92ef24c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:01:18.188128 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1090be-77d8-4c8c-9d19-492da92ef24c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:01:18.189745 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1090be-77d8-4c8c-9d19-492da92ef24c service nova] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:01:18.199839 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1423/2846] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 07:01:18 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:01:24.921691 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=6 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:01:28.195264 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=6 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:01:37.546628 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3905ce6-e3ad-4a88-9d19-9501fed40e6d service placement] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:01:37.556920 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3905ce6-e3ad-4a88-9d19-9501fed40e6d service placement] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:01:37.560769 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2faebe29-c4cd-4a88-a031-080cd9290d2d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bb6c79db75e4d169331c02133d1ec32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:01:37.561377 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2faebe29-c4cd-4a88-a031-080cd9290d2d service neutron] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:01:37.561662 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2faebe29-c4cd-4a88-a031-080cd9290d2d service neutron] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:01:37.562019 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2faebe29-c4cd-4a88-a031-080cd9290d2d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:01:37.563773 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-2faebe29-c4cd-4a88-a031-080cd9290d2d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:01:37.650237 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2faebe29-c4cd-4a88-a031-080cd9290d2d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:01:37.651965 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2faebe29-c4cd-4a88-a031-080cd9290d2d service neutron] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:01:37.678280 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1424/2847] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 07:01:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 07:01:41.289466 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1090be-77d8-4c8c-9d19-492da92ef24c service nova] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:01:41.301247 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1090be-77d8-4c8c-9d19-492da92ef24c service nova] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:01:41.306523 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348b1a48-0ce1-4a7d-86e9-8aca9cd8e103 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abe9d8c9291845dfa4ef55034e5c9e32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:01:41.307121 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348b1a48-0ce1-4a7d-86e9-8aca9cd8e103 service placement] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:01:41.307399 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348b1a48-0ce1-4a7d-86e9-8aca9cd8e103 service placement] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:01:41.307670 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348b1a48-0ce1-4a7d-86e9-8aca9cd8e103 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:01:41.309436 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-348b1a48-0ce1-4a7d-86e9-8aca9cd8e103 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:01:41.321333 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348b1a48-0ce1-4a7d-86e9-8aca9cd8e103 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:01:41.322956 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348b1a48-0ce1-4a7d-86e9-8aca9cd8e103 service placement] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:01:41.335034 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1424/2848] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 07:01:41 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:01:41.463599 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2faebe29-c4cd-4a88-a031-080cd9290d2d service neutron] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:01:41.465565 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-2faebe29-c4cd-4a88-a031-080cd9290d2d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:01:41.479950 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2faebe29-c4cd-4a88-a031-080cd9290d2d service neutron] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:01:41.484666 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b07ad7e-378c-41f6-a31d-27291603ba3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:01:41.485294 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b07ad7e-378c-41f6-a31d-27291603ba3b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:01:41.485572 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b07ad7e-378c-41f6-a31d-27291603ba3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:01:41.485843 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b07ad7e-378c-41f6-a31d-27291603ba3b admin admin] PATH_INFO: `/v3/users/6abfd0a28f9e482badae7018f1b7f9a1` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:01:41.486956 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b07ad7e-378c-41f6-a31d-27291603ba3b admin admin] RBAC: Authorizing `identity:delete_user(user_id=6abfd0a28f9e482badae7018f1b7f9a1)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:01:41.488928 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b07ad7e-378c-41f6-a31d-27291603ba3b admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:01:41.699567 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-0b07ad7e-378c-41f6-a31d-27291603ba3b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6abfd0a28f9e482badae7018f1b7f9a1'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:01:41.707726 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-0b07ad7e-378c-41f6-a31d-27291603ba3b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6abfd0a28f9e482badae7018f1b7f9a1'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:01:41.708363 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-0b07ad7e-378c-41f6-a31d-27291603ba3b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6abfd0a28f9e482badae7018f1b7f9a1'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:01:41.709056 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-0b07ad7e-378c-41f6-a31d-27291603ba3b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6abfd0a28f9e482badae7018f1b7f9a1'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:01:41.804697 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-0b07ad7e-378c-41f6-a31d-27291603ba3b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6abfd0a28f9e482badae7018f1b7f9a1'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:01:41.823480 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1425/2849] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Aug 24 07:01:41 2022] DELETE /identity/v3/users/6abfd0a28f9e482badae7018f1b7f9a1 => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:01:42.172403 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348b1a48-0ce1-4a7d-86e9-8aca9cd8e103 service placement] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:01:42.187343 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348b1a48-0ce1-4a7d-86e9-8aca9cd8e103 service placement] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:01:42.246476 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f4e450-d41d-42d6-a8c0-ebaef29e6a36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:01:42.247121 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f4e450-d41d-42d6-a8c0-ebaef29e6a36 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:01:42.247395 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f4e450-d41d-42d6-a8c0-ebaef29e6a36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:01:42.247691 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f4e450-d41d-42d6-a8c0-ebaef29e6a36 admin admin] PATH_INFO: `/v3/projects/b1f2c78322cb4c999f1e01b82b81a884` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:01:42.249060 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f4e450-d41d-42d6-a8c0-ebaef29e6a36 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b1f2c78322cb4c999f1e01b82b81a884)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:01:42.251662 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f4e450-d41d-42d6-a8c0-ebaef29e6a36 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:01:42.421657 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-25f4e450-d41d-42d6-a8c0-ebaef29e6a36 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b1f2c78322cb4c999f1e01b82b81a884', 'request_id': 'req-25f4e450-d41d-42d6-a8c0-ebaef29e6a36'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:01:42.428294 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-25f4e450-d41d-42d6-a8c0-ebaef29e6a36 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b1f2c78322cb4c999f1e01b82b81a884', 'request_id': 'req-25f4e450-d41d-42d6-a8c0-ebaef29e6a36'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:01:42.435217 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-25f4e450-d41d-42d6-a8c0-ebaef29e6a36 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b1f2c78322cb4c999f1e01b82b81a884', 'request_id': 'req-25f4e450-d41d-42d6-a8c0-ebaef29e6a36'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:01:42.530529 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-25f4e450-d41d-42d6-a8c0-ebaef29e6a36 admin admin] The token cache is being invalidate because project b1f2c78322cb4c999f1e01b82b81a884 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81852) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 07:01:42.530818 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-25f4e450-d41d-42d6-a8c0-ebaef29e6a36 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:01:42.531446 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-25f4e450-d41d-42d6-a8c0-ebaef29e6a36 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:01:42.533118 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1425/2850] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Aug 24 07:01:42 2022] DELETE /identity/v3/projects/b1f2c78322cb4c999f1e01b82b81a884 => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:01:42.540074 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948bdcd8-2558-4363-9f41-d0c69c1355b2 None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:01:42.540634 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948bdcd8-2558-4363-9f41-d0c69c1355b2 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:01:42.540745 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948bdcd8-2558-4363-9f41-d0c69c1355b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:01:42.603368 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:01:42.606200 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-948bdcd8-2558-4363-9f41-d0c69c1355b2 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:01:42.607419 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-948bdcd8-2558-4363-9f41-d0c69c1355b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:01:42.669412 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1426/2851] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 07:01:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:01:42.679661 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d53121f-a824-44e6-8fc5-84ca64f33e0b None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:01:42.679860 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d53121f-a824-44e6-8fc5-84ca64f33e0b None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:01:42.680117 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d53121f-a824-44e6-8fc5-84ca64f33e0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:01:42.703249 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-8d53121f-a824-44e6-8fc5-84ca64f33e0b None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:01:42.704662 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-8d53121f-a824-44e6-8fc5-84ca64f33e0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:01:42.712933 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1426/2852] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 07:01:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:01:42.717465 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948bdcd8-2558-4363-9f41-d0c69c1355b2 None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:01:42.727670 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948bdcd8-2558-4363-9f41-d0c69c1355b2 None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:01:42.733307 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79cd332-6554-4b70-8d1e-38c710cc944d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:01:42.733924 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79cd332-6554-4b70-8d1e-38c710cc944d admin admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:01:42.734236 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79cd332-6554-4b70-8d1e-38c710cc944d admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:01:42.734515 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79cd332-6554-4b70-8d1e-38c710cc944d admin admin] PATH_INFO: `/v3/domains` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:01:42.735170 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79cd332-6554-4b70-8d1e-38c710cc944d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:01:42.735497 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79cd332-6554-4b70-8d1e-38c710cc944d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:01:42.737651 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79cd332-6554-4b70-8d1e-38c710cc944d admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:01:42.750433 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1427/2853] 10.222.0.62 () {58 vars in 1153 bytes} [Wed Aug 24 07:01:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:01:42.755577 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d53121f-a824-44e6-8fc5-84ca64f33e0b None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:01:42.765007 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d53121f-a824-44e6-8fc5-84ca64f33e0b None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:01:42.769624 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ccfc7f-7d4b-4507-96d1-c745e8546170 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:01:42.770224 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ccfc7f-7d4b-4507-96d1-c745e8546170 admin admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:01:42.770503 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ccfc7f-7d4b-4507-96d1-c745e8546170 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:01:42.770779 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ccfc7f-7d4b-4507-96d1-c745e8546170 admin admin] PATH_INFO: `/v3/projects` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:01:42.771408 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ccfc7f-7d4b-4507-96d1-c745e8546170 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:01:42.773373 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ccfc7f-7d4b-4507-96d1-c745e8546170 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:01:42.960895 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1427/2854] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Aug 24 07:01:42 2022] POST /identity/v3/projects => generated 363 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 07:01:42.964981 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79cd332-6554-4b70-8d1e-38c710cc944d admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:01:42.975803 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79cd332-6554-4b70-8d1e-38c710cc944d admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:01:43.030713 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b3c19d-5950-4758-8b38-cd05a1e4774c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:01:43.031315 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b3c19d-5950-4758-8b38-cd05a1e4774c admin admin] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:01:43.031590 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b3c19d-5950-4758-8b38-cd05a1e4774c admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:01:43.031858 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b3c19d-5950-4758-8b38-cd05a1e4774c admin admin] PATH_INFO: `/v3/users` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:01:43.032484 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b3c19d-5950-4758-8b38-cd05a1e4774c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:01:43.034940 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b3c19d-5950-4758-8b38-cd05a1e4774c admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:01:43.130668 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1428/2855] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Aug 24 07:01:42 2022] POST /identity/v3/users => generated 282 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:01:43.135024 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ccfc7f-7d4b-4507-96d1-c745e8546170 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:01:43.145538 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ccfc7f-7d4b-4507-96d1-c745e8546170 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:01:43.150099 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b38946-1c5e-414d-9089-0ef508c54728 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:01:43.150692 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b38946-1c5e-414d-9089-0ef508c54728 admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:01:43.150967 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b38946-1c5e-414d-9089-0ef508c54728 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:01:43.151242 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b38946-1c5e-414d-9089-0ef508c54728 admin admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:01:43.151874 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b38946-1c5e-414d-9089-0ef508c54728 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:01:43.153496 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b38946-1c5e-414d-9089-0ef508c54728 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:01:43.163838 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1428/2856] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Aug 24 07:01:43 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:01:43.168104 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b3c19d-5950-4758-8b38-cd05a1e4774c admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:01:43.178836 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b3c19d-5950-4758-8b38-cd05a1e4774c admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:01:43.183457 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb926321-1af8-44d2-aab1-3073a8dfd932 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:01:43.184049 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb926321-1af8-44d2-aab1-3073a8dfd932 admin admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:01:43.184326 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb926321-1af8-44d2-aab1-3073a8dfd932 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:01:43.184634 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb926321-1af8-44d2-aab1-3073a8dfd932 admin admin] PATH_INFO: `/v3/roles` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:01:43.185400 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb926321-1af8-44d2-aab1-3073a8dfd932 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:01:43.186935 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb926321-1af8-44d2-aab1-3073a8dfd932 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:01:43.198589 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1429/2857] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Aug 24 07:01:43 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:01:43.203048 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b38946-1c5e-414d-9089-0ef508c54728 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:01:43.214036 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b38946-1c5e-414d-9089-0ef508c54728 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:01:43.218741 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10cd480-0cb6-4624-a6d0-bc991060a618 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:01:43.219389 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10cd480-0cb6-4624-a6d0-bc991060a618 admin admin] REQUEST_METHOD: `PUT` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:01:43.219668 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10cd480-0cb6-4624-a6d0-bc991060a618 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:01:43.219943 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10cd480-0cb6-4624-a6d0-bc991060a618 admin admin] PATH_INFO: `/v3/projects/5d205b1ee1fa4d928e3256ecb6315409/users/9ce3b3145d8940878b8da94881c4e8a9/roles/580aa7dbb38d4cff8f9d1bb8983f99ef` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:01:43.234948 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10cd480-0cb6-4624-a6d0-bc991060a618 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d205b1ee1fa4d928e3256ecb6315409, user_id=9ce3b3145d8940878b8da94881c4e8a9, role_id=580aa7dbb38d4cff8f9d1bb8983f99ef)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:01:43.237677 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10cd480-0cb6-4624-a6d0-bc991060a618 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:01:43.331952 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1429/2858] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Aug 24 07:01:43 2022] PUT /identity/v3/projects/5d205b1ee1fa4d928e3256ecb6315409/users/9ce3b3145d8940878b8da94881c4e8a9/roles/580aa7dbb38d4cff8f9d1bb8983f99ef => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:01:43.342653 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1056fd6-bd94-42c9-b237-e385f806adf5 None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:01:43.343064 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1056fd6-bd94-42c9-b237-e385f806adf5 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:01:43.343352 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1056fd6-bd94-42c9-b237-e385f806adf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:01:43.373075 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-c1056fd6-bd94-42c9-b237-e385f806adf5 None None] MFA Rules not processed for user `9ce3b3145d8940878b8da94881c4e8a9`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:01:43.374294 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-c1056fd6-bd94-42c9-b237-e385f806adf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:01:43.441045 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1430/2859] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 07:01:43 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:01:43.445546 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10cd480-0cb6-4624-a6d0-bc991060a618 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:01:43.451515 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:01:43.514311 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10cd480-0cb6-4624-a6d0-bc991060a618 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:01:43.519739 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0393fe-5cc3-447c-a4a0-0b439aaba8b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:01:43.520647 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0393fe-5cc3-447c-a4a0-0b439aaba8b8 admin admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:01:43.520647 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0393fe-5cc3-447c-a4a0-0b439aaba8b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:01:43.520956 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0393fe-5cc3-447c-a4a0-0b439aaba8b8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:01:43.521593 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0393fe-5cc3-447c-a4a0-0b439aaba8b8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:01:43.523795 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0393fe-5cc3-447c-a4a0-0b439aaba8b8 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:01:43.670278 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1430/2860] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Aug 24 07:01:43 2022] POST /identity/v3/projects => generated 365 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:01:43.674992 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1056fd6-bd94-42c9-b237-e385f806adf5 None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:01:43.680698 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:01:43.686972 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1056fd6-bd94-42c9-b237-e385f806adf5 None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:01:43.753030 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a02f4bb-4b1e-447d-a5a2-2b8553ece977 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:01:43.753752 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a02f4bb-4b1e-447d-a5a2-2b8553ece977 admin admin] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:01:43.754091 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a02f4bb-4b1e-447d-a5a2-2b8553ece977 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:01:43.754768 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a02f4bb-4b1e-447d-a5a2-2b8553ece977 admin admin] PATH_INFO: `/v3/users` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:01:43.755764 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a02f4bb-4b1e-447d-a5a2-2b8553ece977 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:01:43.757482 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a02f4bb-4b1e-447d-a5a2-2b8553ece977 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:01:43.885465 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1431/2861] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Aug 24 07:01:43 2022] POST /identity/v3/users => generated 289 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:01:43.889533 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0393fe-5cc3-447c-a4a0-0b439aaba8b8 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:01:43.899893 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0393fe-5cc3-447c-a4a0-0b439aaba8b8 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:01:43.904438 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c76058b-c84f-43b6-bb62-374ba8cc4d6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:01:43.905069 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c76058b-c84f-43b6-bb62-374ba8cc4d6b admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:01:43.905346 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c76058b-c84f-43b6-bb62-374ba8cc4d6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:01:43.905620 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c76058b-c84f-43b6-bb62-374ba8cc4d6b admin admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:01:43.906262 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c76058b-c84f-43b6-bb62-374ba8cc4d6b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:01:43.907845 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c76058b-c84f-43b6-bb62-374ba8cc4d6b admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:01:43.918949 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1431/2862] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Aug 24 07:01:43 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:01:43.923874 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a02f4bb-4b1e-447d-a5a2-2b8553ece977 admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:01:43.934752 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a02f4bb-4b1e-447d-a5a2-2b8553ece977 admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:01:43.941058 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541eaeb6-1d2d-4dbc-a4b8-9068e8c21395 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:01:43.942121 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541eaeb6-1d2d-4dbc-a4b8-9068e8c21395 admin admin] REQUEST_METHOD: `PUT` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:01:43.942533 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541eaeb6-1d2d-4dbc-a4b8-9068e8c21395 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:01:43.942709 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541eaeb6-1d2d-4dbc-a4b8-9068e8c21395 admin admin] PATH_INFO: `/v3/projects/59b26d56066046888f72b04391ecaf33/users/12c37a78d71d456389c749178831159e/roles/49e47cd157d743f6905f1db07ec878a7` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:01:43.958761 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541eaeb6-1d2d-4dbc-a4b8-9068e8c21395 admin admin] RBAC: Authorizing `identity:create_grant(project_id=59b26d56066046888f72b04391ecaf33, user_id=12c37a78d71d456389c749178831159e, role_id=49e47cd157d743f6905f1db07ec878a7)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:01:43.961473 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541eaeb6-1d2d-4dbc-a4b8-9068e8c21395 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:01:44.056235 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1432/2863] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Aug 24 07:01:43 2022] PUT /identity/v3/projects/59b26d56066046888f72b04391ecaf33/users/12c37a78d71d456389c749178831159e/roles/49e47cd157d743f6905f1db07ec878a7 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:01:44.068295 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c76058b-c84f-43b6-bb62-374ba8cc4d6b admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:01:44.079088 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c76058b-c84f-43b6-bb62-374ba8cc4d6b admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:01:44.142109 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be4491c-0f65-4303-8d86-a07103ea6299 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:01:44.142629 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be4491c-0f65-4303-8d86-a07103ea6299 admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:01:44.143402 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be4491c-0f65-4303-8d86-a07103ea6299 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:01:44.143670 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be4491c-0f65-4303-8d86-a07103ea6299 admin admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:01:44.144299 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be4491c-0f65-4303-8d86-a07103ea6299 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:01:44.145921 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be4491c-0f65-4303-8d86-a07103ea6299 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:01:44.155728 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1432/2864] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Aug 24 07:01:44 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:01:44.160616 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541eaeb6-1d2d-4dbc-a4b8-9068e8c21395 admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:01:44.170878 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541eaeb6-1d2d-4dbc-a4b8-9068e8c21395 admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:01:44.175452 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142a6ff6-b03b-4924-a797-fd5f521369d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:01:44.176093 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142a6ff6-b03b-4924-a797-fd5f521369d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:01:44.176372 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142a6ff6-b03b-4924-a797-fd5f521369d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:01:44.176690 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142a6ff6-b03b-4924-a797-fd5f521369d8 admin admin] PATH_INFO: `/v3/projects/59b26d56066046888f72b04391ecaf33/users/12c37a78d71d456389c749178831159e/roles/49e47cd157d743f6905f1db07ec878a7` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:01:44.178586 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142a6ff6-b03b-4924-a797-fd5f521369d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=59b26d56066046888f72b04391ecaf33, user_id=12c37a78d71d456389c749178831159e, role_id=49e47cd157d743f6905f1db07ec878a7)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:01:44.181290 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142a6ff6-b03b-4924-a797-fd5f521369d8 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:01:44.191724 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1433/2865] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Aug 24 07:01:44 2022] PUT /identity/v3/projects/59b26d56066046888f72b04391ecaf33/users/12c37a78d71d456389c749178831159e/roles/49e47cd157d743f6905f1db07ec878a7 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:01:44.202002 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29065b15-50b7-4dd9-bf0f-74026f738022 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:01:44.202284 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29065b15-50b7-4dd9-bf0f-74026f738022 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:01:44.202557 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29065b15-50b7-4dd9-bf0f-74026f738022 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:01:44.233809 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-29065b15-50b7-4dd9-bf0f-74026f738022 None None] MFA Rules not processed for user `12c37a78d71d456389c749178831159e`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:01:44.235201 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-29065b15-50b7-4dd9-bf0f-74026f738022 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:01:44.309314 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:01:44.313436 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1433/2866] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 07:01:44 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:01:44.325294 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142a6ff6-b03b-4924-a797-fd5f521369d8 admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:01:44.327277 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-142a6ff6-b03b-4924-a797-fd5f521369d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:01:44.390326 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142a6ff6-b03b-4924-a797-fd5f521369d8 admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:01:44.394785 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8099784-fc81-4a91-8b3e-18bc93ff9110 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c4def55ac0e48b29836bc5c64bc7402', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:01:44.395338 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8099784-fc81-4a91-8b3e-18bc93ff9110 service cinder] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:01:44.395612 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8099784-fc81-4a91-8b3e-18bc93ff9110 service cinder] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:01:44.395882 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8099784-fc81-4a91-8b3e-18bc93ff9110 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:01:44.405726 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8099784-fc81-4a91-8b3e-18bc93ff9110 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:01:44.407189 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8099784-fc81-4a91-8b3e-18bc93ff9110 service cinder] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:01:44.418771 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1434/2867] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 07:01:44 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:01:44.717577 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29065b15-50b7-4dd9-bf0f-74026f738022 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:01:44.728404 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29065b15-50b7-4dd9-bf0f-74026f738022 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:01:44.732453 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431b7973-586c-4d25-94f5-949aa9471918 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c4def55ac0e48b29836bc5c64bc7402', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:01:44.733115 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431b7973-586c-4d25-94f5-949aa9471918 service cinder] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:01:44.733396 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431b7973-586c-4d25-94f5-949aa9471918 service cinder] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:01:44.733669 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431b7973-586c-4d25-94f5-949aa9471918 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:01:44.797631 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431b7973-586c-4d25-94f5-949aa9471918 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:01:44.799159 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431b7973-586c-4d25-94f5-949aa9471918 service cinder] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:01:44.808743 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1434/2868] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 07:01:44 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:01:48.672889 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8099784-fc81-4a91-8b3e-18bc93ff9110 service cinder] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:01:48.741374 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:01:48.745950 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8099784-fc81-4a91-8b3e-18bc93ff9110 service cinder] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:01:48.750588 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950b6118-f2e4-4c4d-838b-a0fe637bc9b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:01:48.751250 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950b6118-f2e4-4c4d-838b-a0fe637bc9b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:01:48.751523 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950b6118-f2e4-4c4d-838b-a0fe637bc9b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:01:48.751843 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950b6118-f2e4-4c4d-838b-a0fe637bc9b4 admin admin] PATH_INFO: `/v3/users/9ce3b3145d8940878b8da94881c4e8a9` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:01:48.753578 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950b6118-f2e4-4c4d-838b-a0fe637bc9b4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9ce3b3145d8940878b8da94881c4e8a9)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:01:48.755500 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950b6118-f2e4-4c4d-838b-a0fe637bc9b4 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:01:49.003347 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-950b6118-f2e4-4c4d-838b-a0fe637bc9b4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9ce3b3145d8940878b8da94881c4e8a9'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:01:49.011267 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-950b6118-f2e4-4c4d-838b-a0fe637bc9b4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9ce3b3145d8940878b8da94881c4e8a9'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:01:49.012009 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-950b6118-f2e4-4c4d-838b-a0fe637bc9b4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9ce3b3145d8940878b8da94881c4e8a9'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:01:49.012699 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-950b6118-f2e4-4c4d-838b-a0fe637bc9b4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9ce3b3145d8940878b8da94881c4e8a9'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:01:49.130308 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-950b6118-f2e4-4c4d-838b-a0fe637bc9b4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9ce3b3145d8940878b8da94881c4e8a9'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:01:49.149459 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1435/2869] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Aug 24 07:01:48 2022] DELETE /identity/v3/users/9ce3b3145d8940878b8da94881c4e8a9 => generated 0 bytes in 477 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:01:49.154115 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8753b3d9-d669-49f4-8db9-58adb7db39b7 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:01:49.154650 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8753b3d9-d669-49f4-8db9-58adb7db39b7 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:01:49.154650 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8753b3d9-d669-49f4-8db9-58adb7db39b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:01:49.175798 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-8753b3d9-d669-49f4-8db9-58adb7db39b7 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:01:49.177042 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-8753b3d9-d669-49f4-8db9-58adb7db39b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:01:49.239648 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1435/2870] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 07:01:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:01:49.249976 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950b6118-f2e4-4c4d-838b-a0fe637bc9b4 admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:01:49.251395 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-950b6118-f2e4-4c4d-838b-a0fe637bc9b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:01:49.310666 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950b6118-f2e4-4c4d-838b-a0fe637bc9b4 admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:01:49.314323 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0999de4f-afbd-49ea-90fe-47e1aa02595b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bb6c79db75e4d169331c02133d1ec32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:01:49.315463 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0999de4f-afbd-49ea-90fe-47e1aa02595b service neutron] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:01:49.315732 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0999de4f-afbd-49ea-90fe-47e1aa02595b service neutron] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:01:49.315998 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0999de4f-afbd-49ea-90fe-47e1aa02595b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:01:49.326038 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0999de4f-afbd-49ea-90fe-47e1aa02595b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:01:49.327487 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0999de4f-afbd-49ea-90fe-47e1aa02595b service neutron] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:01:49.338458 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1436/2871] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 07:01:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:01:50.072562 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8753b3d9-d669-49f4-8db9-58adb7db39b7 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:01:50.074539 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-8753b3d9-d669-49f4-8db9-58adb7db39b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:01:50.090982 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8753b3d9-d669-49f4-8db9-58adb7db39b7 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:01:50.096294 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e61dbcd-5d8c-4116-b3b2-b25151d93398 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:01:50.096982 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e61dbcd-5d8c-4116-b3b2-b25151d93398 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:01:50.097262 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e61dbcd-5d8c-4116-b3b2-b25151d93398 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:01:50.097525 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e61dbcd-5d8c-4116-b3b2-b25151d93398 admin admin] PATH_INFO: `/v3/projects/5d205b1ee1fa4d928e3256ecb6315409` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:01:50.098679 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e61dbcd-5d8c-4116-b3b2-b25151d93398 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5d205b1ee1fa4d928e3256ecb6315409)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:01:50.100691 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e61dbcd-5d8c-4116-b3b2-b25151d93398 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:01:50.258153 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-4e61dbcd-5d8c-4116-b3b2-b25151d93398 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5d205b1ee1fa4d928e3256ecb6315409', 'request_id': 'req-4e61dbcd-5d8c-4116-b3b2-b25151d93398'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:01:50.263907 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-4e61dbcd-5d8c-4116-b3b2-b25151d93398 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5d205b1ee1fa4d928e3256ecb6315409', 'request_id': 'req-4e61dbcd-5d8c-4116-b3b2-b25151d93398'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:01:50.271631 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-4e61dbcd-5d8c-4116-b3b2-b25151d93398 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5d205b1ee1fa4d928e3256ecb6315409', 'request_id': 'req-4e61dbcd-5d8c-4116-b3b2-b25151d93398'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:01:50.354436 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-4e61dbcd-5d8c-4116-b3b2-b25151d93398 admin admin] The token cache is being invalidate because project 5d205b1ee1fa4d928e3256ecb6315409 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81852) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 07:01:50.354929 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-4e61dbcd-5d8c-4116-b3b2-b25151d93398 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:01:50.355458 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-4e61dbcd-5d8c-4116-b3b2-b25151d93398 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:01:50.357233 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1436/2872] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Aug 24 07:01:50 2022] DELETE /identity/v3/projects/5d205b1ee1fa4d928e3256ecb6315409 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:01:50.362241 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0999de4f-afbd-49ea-90fe-47e1aa02595b service neutron] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:01:50.374974 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0999de4f-afbd-49ea-90fe-47e1aa02595b service neutron] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:01:50.452837 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fa3630-de9a-4427-a326-185165deff2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:01:50.453717 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fa3630-de9a-4427-a326-185165deff2c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:01:50.454450 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fa3630-de9a-4427-a326-185165deff2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:01:50.454989 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fa3630-de9a-4427-a326-185165deff2c admin admin] PATH_INFO: `/v3/users/12c37a78d71d456389c749178831159e` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:01:50.458080 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10fa3630-de9a-4427-a326-185165deff2c admin admin] RBAC: Authorizing `identity:delete_user(user_id=12c37a78d71d456389c749178831159e)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:01:50.462276 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10fa3630-de9a-4427-a326-185165deff2c admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:01:50.506588 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:01:50.715113 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-10fa3630-de9a-4427-a326-185165deff2c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '12c37a78d71d456389c749178831159e'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:01:50.723308 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-10fa3630-de9a-4427-a326-185165deff2c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '12c37a78d71d456389c749178831159e'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:01:50.724314 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-10fa3630-de9a-4427-a326-185165deff2c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '12c37a78d71d456389c749178831159e'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:01:50.725124 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-10fa3630-de9a-4427-a326-185165deff2c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '12c37a78d71d456389c749178831159e'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:01:50.867301 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-10fa3630-de9a-4427-a326-185165deff2c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '12c37a78d71d456389c749178831159e'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:01:50.886401 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1437/2873] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Aug 24 07:01:50 2022] DELETE /identity/v3/users/12c37a78d71d456389c749178831159e => generated 0 bytes in 525 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 07:01:51.435301 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e61dbcd-5d8c-4116-b3b2-b25151d93398 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:01:51.450204 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-4e61dbcd-5d8c-4116-b3b2-b25151d93398 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:01:51.452394 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e61dbcd-5d8c-4116-b3b2-b25151d93398 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:01:51.481847 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:01:51.524183 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef6b816-ab01-4e5c-893b-24de312daee2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:01:51.524774 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef6b816-ab01-4e5c-893b-24de312daee2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:01:51.525063 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef6b816-ab01-4e5c-893b-24de312daee2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:01:51.525338 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef6b816-ab01-4e5c-893b-24de312daee2 admin admin] PATH_INFO: `/v3/projects/59b26d56066046888f72b04391ecaf33` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:01:51.526492 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef6b816-ab01-4e5c-893b-24de312daee2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=59b26d56066046888f72b04391ecaf33)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:01:51.528417 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef6b816-ab01-4e5c-893b-24de312daee2 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:01:51.709867 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-6ef6b816-ab01-4e5c-893b-24de312daee2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '59b26d56066046888f72b04391ecaf33', 'request_id': 'req-6ef6b816-ab01-4e5c-893b-24de312daee2'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:01:51.715393 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-6ef6b816-ab01-4e5c-893b-24de312daee2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '59b26d56066046888f72b04391ecaf33', 'request_id': 'req-6ef6b816-ab01-4e5c-893b-24de312daee2'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:01:51.722970 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-6ef6b816-ab01-4e5c-893b-24de312daee2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '59b26d56066046888f72b04391ecaf33', 'request_id': 'req-6ef6b816-ab01-4e5c-893b-24de312daee2'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:01:51.810817 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-6ef6b816-ab01-4e5c-893b-24de312daee2 admin admin] The token cache is being invalidate because project 59b26d56066046888f72b04391ecaf33 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81852) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 07:01:51.811120 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-6ef6b816-ab01-4e5c-893b-24de312daee2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:01:51.811876 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-6ef6b816-ab01-4e5c-893b-24de312daee2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:01:51.813920 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1437/2874] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Aug 24 07:01:51 2022] DELETE /identity/v3/projects/59b26d56066046888f72b04391ecaf33 => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:01:51.819575 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b21dd9-8df2-47aa-8565-5979ce10a1af None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:01:51.820236 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b21dd9-8df2-47aa-8565-5979ce10a1af None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:01:51.820236 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b21dd9-8df2-47aa-8565-5979ce10a1af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:01:51.843504 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-55b21dd9-8df2-47aa-8565-5979ce10a1af None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:01:51.845270 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-55b21dd9-8df2-47aa-8565-5979ce10a1af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:01:51.912558 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1438/2875] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 07:01:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:01:51.922171 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95a3f5a-4cbb-46df-9e7f-268ea1737c92 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:01:51.922456 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95a3f5a-4cbb-46df-9e7f-268ea1737c92 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:01:51.922731 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95a3f5a-4cbb-46df-9e7f-268ea1737c92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:01:51.944998 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-e95a3f5a-4cbb-46df-9e7f-268ea1737c92 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:01:51.946265 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-e95a3f5a-4cbb-46df-9e7f-268ea1737c92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:01:51.954796 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1438/2876] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 07:01:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:01:51.959473 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b21dd9-8df2-47aa-8565-5979ce10a1af None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:01:51.969701 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b21dd9-8df2-47aa-8565-5979ce10a1af None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:01:51.974971 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1611ad1-1ed4-4f90-ad93-33a0787cc60b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:01:51.975588 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1611ad1-1ed4-4f90-ad93-33a0787cc60b admin admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:01:51.975963 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1611ad1-1ed4-4f90-ad93-33a0787cc60b admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:01:51.976134 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1611ad1-1ed4-4f90-ad93-33a0787cc60b admin admin] PATH_INFO: `/v3/domains` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:01:51.976831 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1611ad1-1ed4-4f90-ad93-33a0787cc60b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:01:51.977135 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1611ad1-1ed4-4f90-ad93-33a0787cc60b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:01:51.978746 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1611ad1-1ed4-4f90-ad93-33a0787cc60b admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:01:51.992358 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1439/2877] 10.222.0.62 () {58 vars in 1153 bytes} [Wed Aug 24 07:01:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:01:51.996812 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95a3f5a-4cbb-46df-9e7f-268ea1737c92 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:01:52.006633 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95a3f5a-4cbb-46df-9e7f-268ea1737c92 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:01:52.011427 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b8b1da-8ad2-41bc-9af7-0b60a9a8999c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:01:52.012055 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b8b1da-8ad2-41bc-9af7-0b60a9a8999c admin admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:01:52.012330 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b8b1da-8ad2-41bc-9af7-0b60a9a8999c admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:01:52.012638 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b8b1da-8ad2-41bc-9af7-0b60a9a8999c admin admin] PATH_INFO: `/v3/projects` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:01:52.013265 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b8b1da-8ad2-41bc-9af7-0b60a9a8999c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:01:52.015250 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b8b1da-8ad2-41bc-9af7-0b60a9a8999c admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:01:52.121349 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1439/2878] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Aug 24 07:01:51 2022] POST /identity/v3/projects => generated 385 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:01:52.125647 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1611ad1-1ed4-4f90-ad93-33a0787cc60b admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:01:52.136054 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1611ad1-1ed4-4f90-ad93-33a0787cc60b admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:01:52.205788 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f945840c-2ab1-4d46-abc8-24ec0312f9ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:01:52.206768 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f945840c-2ab1-4d46-abc8-24ec0312f9ff admin admin] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:01:52.207110 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f945840c-2ab1-4d46-abc8-24ec0312f9ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:01:52.207430 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f945840c-2ab1-4d46-abc8-24ec0312f9ff admin admin] PATH_INFO: `/v3/users` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:01:52.208207 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f945840c-2ab1-4d46-abc8-24ec0312f9ff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:01:52.210545 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f945840c-2ab1-4d46-abc8-24ec0312f9ff admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:01:52.314979 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1440/2879] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Aug 24 07:01:52 2022] POST /identity/v3/users => generated 293 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 07:01:52.319221 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b8b1da-8ad2-41bc-9af7-0b60a9a8999c admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:01:52.331733 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b8b1da-8ad2-41bc-9af7-0b60a9a8999c admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:01:52.336605 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090bc343-99af-4795-a548-704fd7c1cc35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:01:52.337217 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090bc343-99af-4795-a548-704fd7c1cc35 admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:01:52.337491 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090bc343-99af-4795-a548-704fd7c1cc35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:01:52.337755 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090bc343-99af-4795-a548-704fd7c1cc35 admin admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:01:52.338389 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090bc343-99af-4795-a548-704fd7c1cc35 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:01:52.339982 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090bc343-99af-4795-a548-704fd7c1cc35 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:01:52.351773 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1440/2880] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Aug 24 07:01:52 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:01:52.359473 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f945840c-2ab1-4d46-abc8-24ec0312f9ff admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:01:52.370954 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f945840c-2ab1-4d46-abc8-24ec0312f9ff admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:01:52.375993 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388dd126-76de-4183-902d-5ffe1de64742 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:01:52.376578 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388dd126-76de-4183-902d-5ffe1de64742 admin admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:01:52.376863 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388dd126-76de-4183-902d-5ffe1de64742 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:01:52.377725 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388dd126-76de-4183-902d-5ffe1de64742 admin admin] PATH_INFO: `/v3/roles` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:01:52.378375 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388dd126-76de-4183-902d-5ffe1de64742 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:01:52.380073 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388dd126-76de-4183-902d-5ffe1de64742 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:01:52.396378 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1441/2881] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Aug 24 07:01:52 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:01:52.400973 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090bc343-99af-4795-a548-704fd7c1cc35 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:01:52.411097 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090bc343-99af-4795-a548-704fd7c1cc35 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:01:52.415700 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60cf212-0711-458a-87e8-47e21914951a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:01:52.416351 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60cf212-0711-458a-87e8-47e21914951a admin admin] REQUEST_METHOD: `PUT` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:01:52.416667 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60cf212-0711-458a-87e8-47e21914951a admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:01:52.416942 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60cf212-0711-458a-87e8-47e21914951a admin admin] PATH_INFO: `/v3/projects/aa9de36d4bc740f4832dfdd9df4c3b4c/users/0dad24758d814281a1abfcc280e33edb/roles/580aa7dbb38d4cff8f9d1bb8983f99ef` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:01:52.432069 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60cf212-0711-458a-87e8-47e21914951a admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa9de36d4bc740f4832dfdd9df4c3b4c, user_id=0dad24758d814281a1abfcc280e33edb, role_id=580aa7dbb38d4cff8f9d1bb8983f99ef)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:01:52.434572 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60cf212-0711-458a-87e8-47e21914951a admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:01:52.548014 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1441/2882] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Aug 24 07:01:52 2022] PUT /identity/v3/projects/aa9de36d4bc740f4832dfdd9df4c3b4c/users/0dad24758d814281a1abfcc280e33edb/roles/580aa7dbb38d4cff8f9d1bb8983f99ef => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:01:52.558915 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa45eca2-0606-4f9e-bc1d-590a0c6298cb None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:01:52.559258 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa45eca2-0606-4f9e-bc1d-590a0c6298cb None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:01:52.559522 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa45eca2-0606-4f9e-bc1d-590a0c6298cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:01:52.568181 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:01:52.588716 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-aa45eca2-0606-4f9e-bc1d-590a0c6298cb None None] MFA Rules not processed for user `0dad24758d814281a1abfcc280e33edb`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:01:52.589936 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-aa45eca2-0606-4f9e-bc1d-590a0c6298cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:01:52.656596 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1442/2883] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 07:01:52 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:01:52.661120 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60cf212-0711-458a-87e8-47e21914951a admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:01:52.713780 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:01:52.727698 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60cf212-0711-458a-87e8-47e21914951a admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:01:52.732575 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adec0521-c483-4066-95e0-184fb58ec9d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:01:52.733178 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adec0521-c483-4066-95e0-184fb58ec9d4 admin admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:01:52.733459 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adec0521-c483-4066-95e0-184fb58ec9d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:01:52.733736 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adec0521-c483-4066-95e0-184fb58ec9d4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:01:52.734370 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adec0521-c483-4066-95e0-184fb58ec9d4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:01:52.736352 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adec0521-c483-4066-95e0-184fb58ec9d4 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:01:52.886090 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1442/2884] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Aug 24 07:01:52 2022] POST /identity/v3/projects => generated 385 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:01:52.890136 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa45eca2-0606-4f9e-bc1d-590a0c6298cb None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:01:52.900959 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa45eca2-0606-4f9e-bc1d-590a0c6298cb None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:01:52.959216 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ea978b-28e0-43b7-9e35-6f1994468eed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:01:52.959835 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ea978b-28e0-43b7-9e35-6f1994468eed admin admin] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:01:52.960119 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ea978b-28e0-43b7-9e35-6f1994468eed admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:01:52.960398 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ea978b-28e0-43b7-9e35-6f1994468eed admin admin] PATH_INFO: `/v3/users` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:01:52.961072 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ea978b-28e0-43b7-9e35-6f1994468eed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:01:52.963012 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ea978b-28e0-43b7-9e35-6f1994468eed admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:01:53.061721 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1443/2885] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Aug 24 07:01:52 2022] POST /identity/v3/users => generated 299 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 07:01:53.066285 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adec0521-c483-4066-95e0-184fb58ec9d4 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:01:53.077027 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adec0521-c483-4066-95e0-184fb58ec9d4 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:01:53.082591 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5204f70-3484-4573-b95e-6eadb424297a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:01:53.083243 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5204f70-3484-4573-b95e-6eadb424297a admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:01:53.083525 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5204f70-3484-4573-b95e-6eadb424297a admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:01:53.083797 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5204f70-3484-4573-b95e-6eadb424297a admin admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:01:53.084435 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5204f70-3484-4573-b95e-6eadb424297a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:01:53.086089 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5204f70-3484-4573-b95e-6eadb424297a admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:01:53.096358 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1443/2886] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Aug 24 07:01:53 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:01:53.100711 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ea978b-28e0-43b7-9e35-6f1994468eed admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:01:53.110757 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ea978b-28e0-43b7-9e35-6f1994468eed admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:01:53.116052 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9be6fe-dc40-47e6-acfe-2712b262dcf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:01:53.116725 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9be6fe-dc40-47e6-acfe-2712b262dcf3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:01:53.117008 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9be6fe-dc40-47e6-acfe-2712b262dcf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:01:53.117281 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9be6fe-dc40-47e6-acfe-2712b262dcf3 admin admin] PATH_INFO: `/v3/projects/01b2f4f9a32d4389a8a6c16a245135da/users/af3ca20714d344ac8f6cc48b41f9dd6a/roles/49e47cd157d743f6905f1db07ec878a7` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:01:53.133304 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9be6fe-dc40-47e6-acfe-2712b262dcf3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=01b2f4f9a32d4389a8a6c16a245135da, user_id=af3ca20714d344ac8f6cc48b41f9dd6a, role_id=49e47cd157d743f6905f1db07ec878a7)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:01:53.135974 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9be6fe-dc40-47e6-acfe-2712b262dcf3 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:01:53.210770 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1444/2887] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Aug 24 07:01:53 2022] PUT /identity/v3/projects/01b2f4f9a32d4389a8a6c16a245135da/users/af3ca20714d344ac8f6cc48b41f9dd6a/roles/49e47cd157d743f6905f1db07ec878a7 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:01:53.214809 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5204f70-3484-4573-b95e-6eadb424297a admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:01:53.225675 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5204f70-3484-4573-b95e-6eadb424297a admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:01:53.282593 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5282d625-28da-4d8d-b089-0cb78d033b6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:01:53.283189 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5282d625-28da-4d8d-b089-0cb78d033b6d admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:01:53.283460 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5282d625-28da-4d8d-b089-0cb78d033b6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:01:53.283726 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5282d625-28da-4d8d-b089-0cb78d033b6d admin admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:01:53.284350 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5282d625-28da-4d8d-b089-0cb78d033b6d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:01:53.285970 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5282d625-28da-4d8d-b089-0cb78d033b6d admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:01:53.295457 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1444/2888] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Aug 24 07:01:53 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:01:53.300133 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9be6fe-dc40-47e6-acfe-2712b262dcf3 admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:01:53.310580 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9be6fe-dc40-47e6-acfe-2712b262dcf3 admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:01:53.315901 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a49873-c2e4-4794-8ee5-5957e1e6dc27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:01:53.316587 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a49873-c2e4-4794-8ee5-5957e1e6dc27 admin admin] REQUEST_METHOD: `PUT` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:01:53.316871 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a49873-c2e4-4794-8ee5-5957e1e6dc27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:01:53.317153 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a49873-c2e4-4794-8ee5-5957e1e6dc27 admin admin] PATH_INFO: `/v3/projects/01b2f4f9a32d4389a8a6c16a245135da/users/af3ca20714d344ac8f6cc48b41f9dd6a/roles/49e47cd157d743f6905f1db07ec878a7` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:01:53.319079 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a49873-c2e4-4794-8ee5-5957e1e6dc27 admin admin] RBAC: Authorizing `identity:create_grant(project_id=01b2f4f9a32d4389a8a6c16a245135da, user_id=af3ca20714d344ac8f6cc48b41f9dd6a, role_id=49e47cd157d743f6905f1db07ec878a7)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:01:53.321786 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a49873-c2e4-4794-8ee5-5957e1e6dc27 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:01:53.333029 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1445/2889] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Aug 24 07:01:53 2022] PUT /identity/v3/projects/01b2f4f9a32d4389a8a6c16a245135da/users/af3ca20714d344ac8f6cc48b41f9dd6a/roles/49e47cd157d743f6905f1db07ec878a7 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:01:53.347003 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e6511a-c8f2-4d58-b46d-1396560ef463 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:01:53.347290 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e6511a-c8f2-4d58-b46d-1396560ef463 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:01:53.347590 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e6511a-c8f2-4d58-b46d-1396560ef463 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:01:53.375354 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-f5e6511a-c8f2-4d58-b46d-1396560ef463 None None] MFA Rules not processed for user `af3ca20714d344ac8f6cc48b41f9dd6a`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:01:53.376591 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-f5e6511a-c8f2-4d58-b46d-1396560ef463 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:01:53.447734 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1445/2890] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 07:01:53 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:01:53.458739 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a49873-c2e4-4794-8ee5-5957e1e6dc27 admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:01:53.460455 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-c4a49873-c2e4-4794-8ee5-5957e1e6dc27 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:01:53.529410 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:01:53.535698 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a49873-c2e4-4794-8ee5-5957e1e6dc27 admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:01:53.539483 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9401087a-0c59-47e6-a98c-86249a8620ac service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c4def55ac0e48b29836bc5c64bc7402', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:01:53.540093 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9401087a-0c59-47e6-a98c-86249a8620ac service cinder] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:01:53.540371 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9401087a-0c59-47e6-a98c-86249a8620ac service cinder] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:01:53.540679 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9401087a-0c59-47e6-a98c-86249a8620ac service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:01:53.550977 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9401087a-0c59-47e6-a98c-86249a8620ac service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:01:53.552925 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9401087a-0c59-47e6-a98c-86249a8620ac service cinder] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:01:53.563055 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1446/2891] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 07:01:53 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:01:54.792638 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e6511a-c8f2-4d58-b46d-1396560ef463 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:01:54.841435 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:01:54.859177 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e6511a-c8f2-4d58-b46d-1396560ef463 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:01:54.897731 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b5f131-9fa7-43ce-b153-dc62fd407a40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:01:54.897731 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b5f131-9fa7-43ce-b153-dc62fd407a40 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:01:54.897731 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b5f131-9fa7-43ce-b153-dc62fd407a40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:01:54.897731 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b5f131-9fa7-43ce-b153-dc62fd407a40 admin admin] PATH_INFO: `/v3/users/0dad24758d814281a1abfcc280e33edb` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:01:54.898139 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b5f131-9fa7-43ce-b153-dc62fd407a40 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0dad24758d814281a1abfcc280e33edb)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:01:54.898139 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b5f131-9fa7-43ce-b153-dc62fd407a40 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:01:55.089063 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-08b5f131-9fa7-43ce-b153-dc62fd407a40 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0dad24758d814281a1abfcc280e33edb'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:01:55.097921 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-08b5f131-9fa7-43ce-b153-dc62fd407a40 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0dad24758d814281a1abfcc280e33edb'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:01:55.098640 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-08b5f131-9fa7-43ce-b153-dc62fd407a40 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0dad24758d814281a1abfcc280e33edb'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:01:55.187525 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-08b5f131-9fa7-43ce-b153-dc62fd407a40 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0dad24758d814281a1abfcc280e33edb'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:01:55.202207 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-08b5f131-9fa7-43ce-b153-dc62fd407a40 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0dad24758d814281a1abfcc280e33edb'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:01:55.209443 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1446/2892] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Aug 24 07:01:54 2022] DELETE /identity/v3/users/0dad24758d814281a1abfcc280e33edb => generated 0 bytes in 417 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:01:55.214121 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e714f6ef-d5b2-4a0b-b2b7-be8147529efe None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:01:55.214601 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e714f6ef-d5b2-4a0b-b2b7-be8147529efe None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:01:55.214934 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e714f6ef-d5b2-4a0b-b2b7-be8147529efe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:01:55.233672 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-e714f6ef-d5b2-4a0b-b2b7-be8147529efe None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:01:55.234935 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-e714f6ef-d5b2-4a0b-b2b7-be8147529efe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:01:55.297402 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1447/2893] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 07:01:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:01:55.308218 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b5f131-9fa7-43ce-b153-dc62fd407a40 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:01:55.309663 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-08b5f131-9fa7-43ce-b153-dc62fd407a40 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:01:55.370022 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b5f131-9fa7-43ce-b153-dc62fd407a40 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:01:55.373660 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a592a38a-e45b-47e1-a542-f563cb8d7270 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bb6c79db75e4d169331c02133d1ec32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:01:55.374271 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a592a38a-e45b-47e1-a542-f563cb8d7270 service neutron] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:01:55.374547 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a592a38a-e45b-47e1-a542-f563cb8d7270 service neutron] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:01:55.374814 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a592a38a-e45b-47e1-a542-f563cb8d7270 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:01:55.385537 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a592a38a-e45b-47e1-a542-f563cb8d7270 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:01:55.388293 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a592a38a-e45b-47e1-a542-f563cb8d7270 service neutron] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:01:55.401768 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1447/2894] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 07:01:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:01:55.864663 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e714f6ef-d5b2-4a0b-b2b7-be8147529efe None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:01:55.866836 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-e714f6ef-d5b2-4a0b-b2b7-be8147529efe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:01:55.880047 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e714f6ef-d5b2-4a0b-b2b7-be8147529efe None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:01:55.885006 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dce0d1a-07fa-4c71-a4ba-30672518898a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:01:55.885664 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dce0d1a-07fa-4c71-a4ba-30672518898a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:01:55.885946 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dce0d1a-07fa-4c71-a4ba-30672518898a admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:01:55.886222 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dce0d1a-07fa-4c71-a4ba-30672518898a admin admin] PATH_INFO: `/v3/projects/aa9de36d4bc740f4832dfdd9df4c3b4c` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:01:55.887399 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dce0d1a-07fa-4c71-a4ba-30672518898a admin admin] RBAC: Authorizing `identity:delete_project(project_id=aa9de36d4bc740f4832dfdd9df4c3b4c)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:01:55.889413 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dce0d1a-07fa-4c71-a4ba-30672518898a admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:01:56.030604 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-5dce0d1a-07fa-4c71-a4ba-30672518898a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aa9de36d4bc740f4832dfdd9df4c3b4c', 'request_id': 'req-5dce0d1a-07fa-4c71-a4ba-30672518898a'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:01:56.037366 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-5dce0d1a-07fa-4c71-a4ba-30672518898a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aa9de36d4bc740f4832dfdd9df4c3b4c', 'request_id': 'req-5dce0d1a-07fa-4c71-a4ba-30672518898a'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:01:56.044104 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-5dce0d1a-07fa-4c71-a4ba-30672518898a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aa9de36d4bc740f4832dfdd9df4c3b4c', 'request_id': 'req-5dce0d1a-07fa-4c71-a4ba-30672518898a'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:01:56.143432 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-5dce0d1a-07fa-4c71-a4ba-30672518898a admin admin] The token cache is being invalidate because project aa9de36d4bc740f4832dfdd9df4c3b4c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81851) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 07:01:56.143678 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-5dce0d1a-07fa-4c71-a4ba-30672518898a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:01:56.144294 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-5dce0d1a-07fa-4c71-a4ba-30672518898a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:01:56.145845 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1448/2895] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Aug 24 07:01:55 2022] DELETE /identity/v3/projects/aa9de36d4bc740f4832dfdd9df4c3b4c => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 07:01:56.149877 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a592a38a-e45b-47e1-a542-f563cb8d7270 service neutron] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:01:56.160519 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a592a38a-e45b-47e1-a542-f563cb8d7270 service neutron] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:01:56.217908 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1365f1-ab15-4691-9634-cac849aeeeb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:01:56.218658 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1365f1-ab15-4691-9634-cac849aeeeb3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:01:56.219007 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1365f1-ab15-4691-9634-cac849aeeeb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:01:56.219317 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1365f1-ab15-4691-9634-cac849aeeeb3 admin admin] PATH_INFO: `/v3/users/af3ca20714d344ac8f6cc48b41f9dd6a` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:01:56.221262 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1365f1-ab15-4691-9634-cac849aeeeb3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=af3ca20714d344ac8f6cc48b41f9dd6a)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:01:56.223243 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1365f1-ab15-4691-9634-cac849aeeeb3 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:01:56.228884 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:01:56.483684 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-ba1365f1-ab15-4691-9634-cac849aeeeb3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'af3ca20714d344ac8f6cc48b41f9dd6a'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:01:56.492225 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-ba1365f1-ab15-4691-9634-cac849aeeeb3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'af3ca20714d344ac8f6cc48b41f9dd6a'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:01:56.492997 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-ba1365f1-ab15-4691-9634-cac849aeeeb3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'af3ca20714d344ac8f6cc48b41f9dd6a'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:01:56.570431 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-ba1365f1-ab15-4691-9634-cac849aeeeb3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'af3ca20714d344ac8f6cc48b41f9dd6a'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:01:56.585084 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-ba1365f1-ab15-4691-9634-cac849aeeeb3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'af3ca20714d344ac8f6cc48b41f9dd6a'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:01:56.592199 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1448/2896] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Aug 24 07:01:56 2022] DELETE /identity/v3/users/af3ca20714d344ac8f6cc48b41f9dd6a => generated 0 bytes in 443 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:01:57.069520 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dce0d1a-07fa-4c71-a4ba-30672518898a admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:01:57.078368 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-5dce0d1a-07fa-4c71-a4ba-30672518898a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:01:57.087044 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dce0d1a-07fa-4c71-a4ba-30672518898a admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:01:57.151272 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2eba18-a343-48bc-b767-1ec55f113c58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:01:57.151917 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2eba18-a343-48bc-b767-1ec55f113c58 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:01:57.152199 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2eba18-a343-48bc-b767-1ec55f113c58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:01:57.152473 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2eba18-a343-48bc-b767-1ec55f113c58 admin admin] PATH_INFO: `/v3/projects/01b2f4f9a32d4389a8a6c16a245135da` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:01:57.153701 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2eba18-a343-48bc-b767-1ec55f113c58 admin admin] RBAC: Authorizing `identity:delete_project(project_id=01b2f4f9a32d4389a8a6c16a245135da)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:01:57.155661 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2eba18-a343-48bc-b767-1ec55f113c58 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:01:57.174113 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:01:57.290203 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-1d2eba18-a343-48bc-b767-1ec55f113c58 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '01b2f4f9a32d4389a8a6c16a245135da', 'request_id': 'req-1d2eba18-a343-48bc-b767-1ec55f113c58'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:01:57.296024 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-1d2eba18-a343-48bc-b767-1ec55f113c58 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '01b2f4f9a32d4389a8a6c16a245135da', 'request_id': 'req-1d2eba18-a343-48bc-b767-1ec55f113c58'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:01:57.303026 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-1d2eba18-a343-48bc-b767-1ec55f113c58 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '01b2f4f9a32d4389a8a6c16a245135da', 'request_id': 'req-1d2eba18-a343-48bc-b767-1ec55f113c58'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:01:57.432615 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-1d2eba18-a343-48bc-b767-1ec55f113c58 admin admin] The token cache is being invalidate because project 01b2f4f9a32d4389a8a6c16a245135da was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81851) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 07:01:57.432909 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-1d2eba18-a343-48bc-b767-1ec55f113c58 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:01:57.433532 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-1d2eba18-a343-48bc-b767-1ec55f113c58 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:01:57.435086 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1449/2897] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Aug 24 07:01:57 2022] DELETE /identity/v3/projects/01b2f4f9a32d4389a8a6c16a245135da => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:01:57.441371 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6daf14-0ee6-4f3c-9f0d-3fe340f47ca2 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:01:57.441903 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6daf14-0ee6-4f3c-9f0d-3fe340f47ca2 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:01:57.441903 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6daf14-0ee6-4f3c-9f0d-3fe340f47ca2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:01:57.462493 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-af6daf14-0ee6-4f3c-9f0d-3fe340f47ca2 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:01:57.463770 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-af6daf14-0ee6-4f3c-9f0d-3fe340f47ca2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:01:57.522941 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1449/2898] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 07:01:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:01:57.530909 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c5d68f-6b00-469b-a110-ae1162a6bae2 None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:01:57.531210 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c5d68f-6b00-469b-a110-ae1162a6bae2 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:01:57.531510 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c5d68f-6b00-469b-a110-ae1162a6bae2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:01:57.552770 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-d7c5d68f-6b00-469b-a110-ae1162a6bae2 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:01:57.554031 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-d7c5d68f-6b00-469b-a110-ae1162a6bae2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:01:57.562144 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1450/2899] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 07:01:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:01:57.566513 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6daf14-0ee6-4f3c-9f0d-3fe340f47ca2 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:01:57.577035 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6daf14-0ee6-4f3c-9f0d-3fe340f47ca2 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:01:57.582432 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632b89e2-eab5-4e54-b7cb-636083ffe6ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:01:57.583049 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632b89e2-eab5-4e54-b7cb-636083ffe6ba admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:01:57.583325 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632b89e2-eab5-4e54-b7cb-636083ffe6ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:01:57.583597 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632b89e2-eab5-4e54-b7cb-636083ffe6ba admin admin] PATH_INFO: `/v3/domains` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:01:57.584243 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632b89e2-eab5-4e54-b7cb-636083ffe6ba admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:01:57.584581 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632b89e2-eab5-4e54-b7cb-636083ffe6ba admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:01:57.586205 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632b89e2-eab5-4e54-b7cb-636083ffe6ba admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:01:57.599630 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1450/2900] 10.222.0.62 () {58 vars in 1153 bytes} [Wed Aug 24 07:01:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:01:57.606715 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c5d68f-6b00-469b-a110-ae1162a6bae2 None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:01:57.618191 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c5d68f-6b00-469b-a110-ae1162a6bae2 None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:01:57.623661 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50bfa59-0419-41f5-ae17-f0933feae2b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:01:57.624263 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50bfa59-0419-41f5-ae17-f0933feae2b3 admin admin] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:01:57.624571 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50bfa59-0419-41f5-ae17-f0933feae2b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:01:57.624846 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50bfa59-0419-41f5-ae17-f0933feae2b3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:01:57.625471 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50bfa59-0419-41f5-ae17-f0933feae2b3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:01:57.627446 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50bfa59-0419-41f5-ae17-f0933feae2b3 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:01:57.728280 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1451/2901] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Aug 24 07:01:57 2022] POST /identity/v3/projects => generated 361 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:01:57.733153 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632b89e2-eab5-4e54-b7cb-636083ffe6ba admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:01:57.743722 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632b89e2-eab5-4e54-b7cb-636083ffe6ba admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:01:57.803616 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f284f927-f7b7-4baf-9d15-f69dbd3c4223 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:01:57.804237 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f284f927-f7b7-4baf-9d15-f69dbd3c4223 admin admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:01:57.804542 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f284f927-f7b7-4baf-9d15-f69dbd3c4223 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:01:57.804819 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f284f927-f7b7-4baf-9d15-f69dbd3c4223 admin admin] PATH_INFO: `/v3/users` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:01:57.805455 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f284f927-f7b7-4baf-9d15-f69dbd3c4223 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:01:57.807399 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f284f927-f7b7-4baf-9d15-f69dbd3c4223 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:01:57.899852 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1451/2902] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Aug 24 07:01:57 2022] POST /identity/v3/users => generated 281 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:01:57.904066 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50bfa59-0419-41f5-ae17-f0933feae2b3 admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:01:57.914834 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50bfa59-0419-41f5-ae17-f0933feae2b3 admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:01:57.920268 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29894412-b8f1-4adc-8bba-243285a9900c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:01:57.920268 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29894412-b8f1-4adc-8bba-243285a9900c admin admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:01:57.920268 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29894412-b8f1-4adc-8bba-243285a9900c admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:01:57.920660 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29894412-b8f1-4adc-8bba-243285a9900c admin admin] PATH_INFO: `/v3/roles` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:01:57.921255 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29894412-b8f1-4adc-8bba-243285a9900c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:01:57.922847 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29894412-b8f1-4adc-8bba-243285a9900c admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:01:57.935091 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1452/2903] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Aug 24 07:01:57 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:01:57.939350 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f284f927-f7b7-4baf-9d15-f69dbd3c4223 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:01:57.945688 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:01:57.951514 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f284f927-f7b7-4baf-9d15-f69dbd3c4223 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:01:57.956260 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be650d2-5f53-4bbe-9350-fd063df2c28b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:01:57.956900 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be650d2-5f53-4bbe-9350-fd063df2c28b admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:01:57.957176 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be650d2-5f53-4bbe-9350-fd063df2c28b admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:01:57.962563 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be650d2-5f53-4bbe-9350-fd063df2c28b admin admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:01:57.962563 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be650d2-5f53-4bbe-9350-fd063df2c28b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:01:57.962563 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be650d2-5f53-4bbe-9350-fd063df2c28b admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:01:57.971871 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1452/2904] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Aug 24 07:01:57 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:01:57.975946 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29894412-b8f1-4adc-8bba-243285a9900c admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:01:57.985943 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29894412-b8f1-4adc-8bba-243285a9900c admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:01:57.991273 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c5926e-58da-40f6-ab23-e0f17c7e5bc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:01:57.991909 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c5926e-58da-40f6-ab23-e0f17c7e5bc3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:01:57.996900 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c5926e-58da-40f6-ab23-e0f17c7e5bc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:01:57.996900 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c5926e-58da-40f6-ab23-e0f17c7e5bc3 admin admin] PATH_INFO: `/v3/projects/8108754605324e54b49419ace40dac4f/users/8d5631dfd7e64cddb56d43996b4ef30d/roles/580aa7dbb38d4cff8f9d1bb8983f99ef` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:01:58.007802 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c5926e-58da-40f6-ab23-e0f17c7e5bc3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8108754605324e54b49419ace40dac4f, user_id=8d5631dfd7e64cddb56d43996b4ef30d, role_id=580aa7dbb38d4cff8f9d1bb8983f99ef)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:01:58.010522 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c5926e-58da-40f6-ab23-e0f17c7e5bc3 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:01:58.085169 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1453/2905] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Aug 24 07:01:57 2022] PUT /identity/v3/projects/8108754605324e54b49419ace40dac4f/users/8d5631dfd7e64cddb56d43996b4ef30d/roles/580aa7dbb38d4cff8f9d1bb8983f99ef => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:01:58.096863 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503b2fa0-63f8-4b75-8328-c8a5d9ed13a4 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:01:58.097147 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503b2fa0-63f8-4b75-8328-c8a5d9ed13a4 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:01:58.097424 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503b2fa0-63f8-4b75-8328-c8a5d9ed13a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:01:58.127142 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-503b2fa0-63f8-4b75-8328-c8a5d9ed13a4 None None] MFA Rules not processed for user `8d5631dfd7e64cddb56d43996b4ef30d`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:01:58.128310 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-503b2fa0-63f8-4b75-8328-c8a5d9ed13a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:01:58.194713 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1453/2906] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 07:01:58 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:01:58.228430 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c5926e-58da-40f6-ab23-e0f17c7e5bc3 admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:01:58.230184 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-f6c5926e-58da-40f6-ab23-e0f17c7e5bc3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:01:58.287460 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c5926e-58da-40f6-ab23-e0f17c7e5bc3 admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:01:58.291158 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4ab3fb-bb4a-47ba-b8a6-61281dcfde26 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c4def55ac0e48b29836bc5c64bc7402', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:01:58.291757 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4ab3fb-bb4a-47ba-b8a6-61281dcfde26 service cinder] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:01:58.292036 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4ab3fb-bb4a-47ba-b8a6-61281dcfde26 service cinder] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:01:58.292305 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4ab3fb-bb4a-47ba-b8a6-61281dcfde26 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:01:58.300920 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4ab3fb-bb4a-47ba-b8a6-61281dcfde26 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:01:58.302347 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4ab3fb-bb4a-47ba-b8a6-61281dcfde26 service cinder] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:01:58.311108 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1454/2907] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 07:01:58 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:01:58.326569 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503b2fa0-63f8-4b75-8328-c8a5d9ed13a4 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:01:58.391408 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503b2fa0-63f8-4b75-8328-c8a5d9ed13a4 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:01:58.395912 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3be2234-ec91-4f40-91d7-bdabf7c80671 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:01:58.396568 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3be2234-ec91-4f40-91d7-bdabf7c80671 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:01:58.396849 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3be2234-ec91-4f40-91d7-bdabf7c80671 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:01:58.397119 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3be2234-ec91-4f40-91d7-bdabf7c80671 admin admin] PATH_INFO: `/v3/users/8d5631dfd7e64cddb56d43996b4ef30d` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:01:58.398247 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3be2234-ec91-4f40-91d7-bdabf7c80671 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8d5631dfd7e64cddb56d43996b4ef30d)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:01:58.400185 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3be2234-ec91-4f40-91d7-bdabf7c80671 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:01:58.607873 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-c3be2234-ec91-4f40-91d7-bdabf7c80671 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8d5631dfd7e64cddb56d43996b4ef30d'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:01:58.614676 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-c3be2234-ec91-4f40-91d7-bdabf7c80671 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8d5631dfd7e64cddb56d43996b4ef30d'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:01:58.615308 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-c3be2234-ec91-4f40-91d7-bdabf7c80671 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8d5631dfd7e64cddb56d43996b4ef30d'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:01:58.701940 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-c3be2234-ec91-4f40-91d7-bdabf7c80671 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8d5631dfd7e64cddb56d43996b4ef30d'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:01:58.715717 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-c3be2234-ec91-4f40-91d7-bdabf7c80671 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8d5631dfd7e64cddb56d43996b4ef30d'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:01:58.717997 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:01:58.722552 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1454/2908] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Aug 24 07:01:58 2022] DELETE /identity/v3/users/8d5631dfd7e64cddb56d43996b4ef30d => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:01:58.726951 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b78a04-f3f1-4847-9d1b-09174ffdff65 None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:01:58.727221 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b78a04-f3f1-4847-9d1b-09174ffdff65 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:01:58.727490 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b78a04-f3f1-4847-9d1b-09174ffdff65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:01:58.746276 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-11b78a04-f3f1-4847-9d1b-09174ffdff65 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:01:58.747489 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-11b78a04-f3f1-4847-9d1b-09174ffdff65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:01:58.785770 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:01:58.806575 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1455/2909] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 07:01:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:01:58.816417 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3be2234-ec91-4f40-91d7-bdabf7c80671 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:01:58.817876 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-c3be2234-ec91-4f40-91d7-bdabf7c80671 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:01:58.875632 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3be2234-ec91-4f40-91d7-bdabf7c80671 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:01:58.880370 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017aa65a-9c31-4900-8680-ba7c5cbc991b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bb6c79db75e4d169331c02133d1ec32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:01:58.881035 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017aa65a-9c31-4900-8680-ba7c5cbc991b service neutron] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:01:58.881315 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017aa65a-9c31-4900-8680-ba7c5cbc991b service neutron] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:01:58.881588 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017aa65a-9c31-4900-8680-ba7c5cbc991b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:01:58.890545 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017aa65a-9c31-4900-8680-ba7c5cbc991b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:01:58.891999 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017aa65a-9c31-4900-8680-ba7c5cbc991b service neutron] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:01:58.902299 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1455/2910] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 07:01:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:01:59.383885 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b78a04-f3f1-4847-9d1b-09174ffdff65 None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:01:59.385370 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-11b78a04-f3f1-4847-9d1b-09174ffdff65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:01:59.399609 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b78a04-f3f1-4847-9d1b-09174ffdff65 None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:01:59.404929 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3f239f-c6a1-4143-b849-edf341fea57c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:01:59.405580 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3f239f-c6a1-4143-b849-edf341fea57c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:01:59.405858 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3f239f-c6a1-4143-b849-edf341fea57c admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:01:59.406130 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3f239f-c6a1-4143-b849-edf341fea57c admin admin] PATH_INFO: `/v3/projects/8108754605324e54b49419ace40dac4f` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:01:59.407368 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3f239f-c6a1-4143-b849-edf341fea57c admin admin] RBAC: Authorizing `identity:delete_project(project_id=8108754605324e54b49419ace40dac4f)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:01:59.410303 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3f239f-c6a1-4143-b849-edf341fea57c admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:01:59.527585 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-eb3f239f-c6a1-4143-b849-edf341fea57c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8108754605324e54b49419ace40dac4f', 'request_id': 'req-eb3f239f-c6a1-4143-b849-edf341fea57c'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:01:59.534394 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-eb3f239f-c6a1-4143-b849-edf341fea57c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8108754605324e54b49419ace40dac4f', 'request_id': 'req-eb3f239f-c6a1-4143-b849-edf341fea57c'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:01:59.541970 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-eb3f239f-c6a1-4143-b849-edf341fea57c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8108754605324e54b49419ace40dac4f', 'request_id': 'req-eb3f239f-c6a1-4143-b849-edf341fea57c'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:01:59.656171 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-eb3f239f-c6a1-4143-b849-edf341fea57c admin admin] The token cache is being invalidate because project 8108754605324e54b49419ace40dac4f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81851) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 07:01:59.656452 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-eb3f239f-c6a1-4143-b849-edf341fea57c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:01:59.657133 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-eb3f239f-c6a1-4143-b849-edf341fea57c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:01:59.658712 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1456/2911] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Aug 24 07:01:59 2022] DELETE /identity/v3/projects/8108754605324e54b49419ace40dac4f => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:01:59.663727 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71753586-561f-408f-af7b-be9f7bb71dba None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:01:59.664264 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71753586-561f-408f-af7b-be9f7bb71dba None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:01:59.664264 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71753586-561f-408f-af7b-be9f7bb71dba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:01:59.684633 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-71753586-561f-408f-af7b-be9f7bb71dba None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:01:59.685842 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-71753586-561f-408f-af7b-be9f7bb71dba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:01:59.749244 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1456/2912] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 07:01:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 07:01:59.758116 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22101f2f-781e-4387-b42c-d5f8ce47a338 None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:01:59.758400 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22101f2f-781e-4387-b42c-d5f8ce47a338 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:01:59.758671 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22101f2f-781e-4387-b42c-d5f8ce47a338 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:01:59.778450 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-22101f2f-781e-4387-b42c-d5f8ce47a338 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:01:59.779644 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-22101f2f-781e-4387-b42c-d5f8ce47a338 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:01:59.788398 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1457/2913] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 07:01:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:01:59.792646 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71753586-561f-408f-af7b-be9f7bb71dba None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:01:59.803047 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71753586-561f-408f-af7b-be9f7bb71dba None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:01:59.807592 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6227ef39-d98a-46a1-92ce-838fbe2db8aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:01:59.808187 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6227ef39-d98a-46a1-92ce-838fbe2db8aa admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:01:59.808465 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6227ef39-d98a-46a1-92ce-838fbe2db8aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:01:59.808770 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6227ef39-d98a-46a1-92ce-838fbe2db8aa admin admin] PATH_INFO: `/v3/domains` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:01:59.809431 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6227ef39-d98a-46a1-92ce-838fbe2db8aa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:01:59.809736 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6227ef39-d98a-46a1-92ce-838fbe2db8aa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:01:59.811337 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6227ef39-d98a-46a1-92ce-838fbe2db8aa admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:01:59.823067 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1457/2914] 10.222.0.62 () {58 vars in 1153 bytes} [Wed Aug 24 07:01:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:01:59.828404 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22101f2f-781e-4387-b42c-d5f8ce47a338 None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:01:59.838793 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22101f2f-781e-4387-b42c-d5f8ce47a338 None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:01:59.845442 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae32ca3-cbd3-4a19-888a-3e2d7d125ab5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:01:59.846718 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae32ca3-cbd3-4a19-888a-3e2d7d125ab5 admin admin] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:01:59.847046 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae32ca3-cbd3-4a19-888a-3e2d7d125ab5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:01:59.847434 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae32ca3-cbd3-4a19-888a-3e2d7d125ab5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:01:59.848574 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae32ca3-cbd3-4a19-888a-3e2d7d125ab5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:01:59.851289 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae32ca3-cbd3-4a19-888a-3e2d7d125ab5 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:01:59.989418 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1458/2915] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Aug 24 07:01:59 2022] POST /identity/v3/projects => generated 371 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:01:59.993982 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6227ef39-d98a-46a1-92ce-838fbe2db8aa admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:02:00.004348 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6227ef39-d98a-46a1-92ce-838fbe2db8aa admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:02:00.063522 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2af158a-8cbd-4974-8f90-92efee974bf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:02:00.064125 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2af158a-8cbd-4974-8f90-92efee974bf0 admin admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:02:00.064973 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2af158a-8cbd-4974-8f90-92efee974bf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:02:00.065248 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2af158a-8cbd-4974-8f90-92efee974bf0 admin admin] PATH_INFO: `/v3/users` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:02:00.065870 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2af158a-8cbd-4974-8f90-92efee974bf0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:02:00.067811 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2af158a-8cbd-4974-8f90-92efee974bf0 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:02:00.171184 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1458/2916] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Aug 24 07:01:59 2022] POST /identity/v3/users => generated 286 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 07:02:00.176972 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae32ca3-cbd3-4a19-888a-3e2d7d125ab5 admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:02:00.188948 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae32ca3-cbd3-4a19-888a-3e2d7d125ab5 admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:02:00.194429 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e83c99-6bc6-4202-b430-fc51304be1ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:02:00.195454 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e83c99-6bc6-4202-b430-fc51304be1ab admin admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:02:00.195585 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e83c99-6bc6-4202-b430-fc51304be1ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:02:00.195762 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e83c99-6bc6-4202-b430-fc51304be1ab admin admin] PATH_INFO: `/v3/roles` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:02:00.196410 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e83c99-6bc6-4202-b430-fc51304be1ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:02:00.198109 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e83c99-6bc6-4202-b430-fc51304be1ab admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:02:00.209306 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1459/2917] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Aug 24 07:02:00 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 07:02:00.213931 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2af158a-8cbd-4974-8f90-92efee974bf0 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:02:00.225385 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2af158a-8cbd-4974-8f90-92efee974bf0 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:02:00.230122 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe9f1d7-f7a3-45c6-ad2f-f846873a2c06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:02:00.230727 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe9f1d7-f7a3-45c6-ad2f-f846873a2c06 admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:02:00.231005 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe9f1d7-f7a3-45c6-ad2f-f846873a2c06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:02:00.231275 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe9f1d7-f7a3-45c6-ad2f-f846873a2c06 admin admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:02:00.231913 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe9f1d7-f7a3-45c6-ad2f-f846873a2c06 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:02:00.233540 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe9f1d7-f7a3-45c6-ad2f-f846873a2c06 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:02:00.245855 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1459/2918] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Aug 24 07:02:00 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:02:00.250238 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e83c99-6bc6-4202-b430-fc51304be1ab admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:02:00.261440 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e83c99-6bc6-4202-b430-fc51304be1ab admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:02:00.266504 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f689c3f0-c90c-408b-be1b-b5445a221e2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:02:00.267185 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f689c3f0-c90c-408b-be1b-b5445a221e2f admin admin] REQUEST_METHOD: `PUT` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:02:00.267497 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f689c3f0-c90c-408b-be1b-b5445a221e2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:02:00.267775 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f689c3f0-c90c-408b-be1b-b5445a221e2f admin admin] PATH_INFO: `/v3/projects/057aefc9bbcc4818ab055d3c0ddacdb5/users/3a2cc6be0c09472a9b3d319e82d95534/roles/580aa7dbb38d4cff8f9d1bb8983f99ef` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:02:00.284565 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f689c3f0-c90c-408b-be1b-b5445a221e2f admin admin] RBAC: Authorizing `identity:create_grant(project_id=057aefc9bbcc4818ab055d3c0ddacdb5, user_id=3a2cc6be0c09472a9b3d319e82d95534, role_id=580aa7dbb38d4cff8f9d1bb8983f99ef)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:02:00.287265 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f689c3f0-c90c-408b-be1b-b5445a221e2f admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:02:00.377361 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1460/2919] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Aug 24 07:02:00 2022] PUT /identity/v3/projects/057aefc9bbcc4818ab055d3c0ddacdb5/users/3a2cc6be0c09472a9b3d319e82d95534/roles/580aa7dbb38d4cff8f9d1bb8983f99ef => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 07:02:00.383207 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6010220a-f909-48d7-bc59-4e679d2f89b4 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:02:00.383489 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6010220a-f909-48d7-bc59-4e679d2f89b4 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:02:00.383768 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6010220a-f909-48d7-bc59-4e679d2f89b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:02:00.403699 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-6010220a-f909-48d7-bc59-4e679d2f89b4 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:02:00.404963 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-6010220a-f909-48d7-bc59-4e679d2f89b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:02:00.409993 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:02:00.472522 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1460/2920] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 07:02:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:02:00.484182 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f689c3f0-c90c-408b-be1b-b5445a221e2f admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:02:00.486169 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-f689c3f0-c90c-408b-be1b-b5445a221e2f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:02:00.549796 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f689c3f0-c90c-408b-be1b-b5445a221e2f admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:02:00.553431 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cfdea9-8e30-4f57-acc1-003ef5c8cc1a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bb6c79db75e4d169331c02133d1ec32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:02:00.554040 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cfdea9-8e30-4f57-acc1-003ef5c8cc1a service neutron] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:02:00.554318 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cfdea9-8e30-4f57-acc1-003ef5c8cc1a service neutron] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:02:00.554586 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cfdea9-8e30-4f57-acc1-003ef5c8cc1a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:02:00.564795 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86cfdea9-8e30-4f57-acc1-003ef5c8cc1a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:02:00.566233 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86cfdea9-8e30-4f57-acc1-003ef5c8cc1a service neutron] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:02:00.576556 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1461/2921] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 07:02:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:02:00.661493 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6010220a-f909-48d7-bc59-4e679d2f89b4 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:02:00.662954 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-6010220a-f909-48d7-bc59-4e679d2f89b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:02:00.676172 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6010220a-f909-48d7-bc59-4e679d2f89b4 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:02:00.679954 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae0872b-7318-4f6c-96ba-a992d6e708d2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bb6c79db75e4d169331c02133d1ec32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:02:00.680604 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae0872b-7318-4f6c-96ba-a992d6e708d2 service neutron] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:02:00.680887 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae0872b-7318-4f6c-96ba-a992d6e708d2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:02:00.681161 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae0872b-7318-4f6c-96ba-a992d6e708d2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:02:00.682952 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-1ae0872b-7318-4f6c-96ba-a992d6e708d2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:02:00.694403 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae0872b-7318-4f6c-96ba-a992d6e708d2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:02:00.695865 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae0872b-7318-4f6c-96ba-a992d6e708d2 service neutron] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:02:00.706059 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1461/2922] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 07:02:00 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:02:02.308334 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650069ef-4fad-4f89-8de1-54ed5937c8c2 None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:02:02.308820 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650069ef-4fad-4f89-8de1-54ed5937c8c2 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:02:02.308960 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650069ef-4fad-4f89-8de1-54ed5937c8c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:02:02.317611 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:02:02.341586 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-650069ef-4fad-4f89-8de1-54ed5937c8c2 None None] MFA Rules not processed for user `3a2cc6be0c09472a9b3d319e82d95534`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:02:02.342805 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-650069ef-4fad-4f89-8de1-54ed5937c8c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:02:02.412029 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1462/2923] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 07:02:02 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:02:02.423304 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae0872b-7318-4f6c-96ba-a992d6e708d2 service neutron] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:02:02.424794 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-1ae0872b-7318-4f6c-96ba-a992d6e708d2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:02:02.488099 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae0872b-7318-4f6c-96ba-a992d6e708d2 service neutron] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:02:02.492275 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5136b2fe-fd6b-4ac5-9c83-575585b9353a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c4def55ac0e48b29836bc5c64bc7402', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:02:02.492954 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5136b2fe-fd6b-4ac5-9c83-575585b9353a service cinder] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:02:02.493230 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5136b2fe-fd6b-4ac5-9c83-575585b9353a service cinder] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:02:02.493504 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5136b2fe-fd6b-4ac5-9c83-575585b9353a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:02:02.503438 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5136b2fe-fd6b-4ac5-9c83-575585b9353a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:02:02.504961 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5136b2fe-fd6b-4ac5-9c83-575585b9353a service cinder] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:02:02.514518 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1462/2924] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 07:02:02 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:02:04.371634 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317b3103-9f96-479f-80cb-55989441cce7 None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:02:04.372148 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317b3103-9f96-479f-80cb-55989441cce7 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:02:04.372148 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317b3103-9f96-479f-80cb-55989441cce7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:02:04.391754 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-317b3103-9f96-479f-80cb-55989441cce7 None None] MFA Rules not processed for user `3a2cc6be0c09472a9b3d319e82d95534`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:02:04.393007 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-317b3103-9f96-479f-80cb-55989441cce7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:02:04.399856 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1463/2925] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 07:02:04 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:02:04.641535 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5136b2fe-fd6b-4ac5-9c83-575585b9353a service cinder] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:02:04.651218 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5136b2fe-fd6b-4ac5-9c83-575585b9353a service cinder] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:02:04.655033 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2691fc-c984-4fff-bdff-62063f741df8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bb6c79db75e4d169331c02133d1ec32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:02:04.655647 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2691fc-c984-4fff-bdff-62063f741df8 service neutron] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:02:04.655957 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2691fc-c984-4fff-bdff-62063f741df8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:02:04.656232 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2691fc-c984-4fff-bdff-62063f741df8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:02:04.658102 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-0e2691fc-c984-4fff-bdff-62063f741df8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:02:04.667888 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2691fc-c984-4fff-bdff-62063f741df8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:02:04.669344 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2691fc-c984-4fff-bdff-62063f741df8 service neutron] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:02:04.677907 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1463/2926] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 07:02:04 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:02:05.197528 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317b3103-9f96-479f-80cb-55989441cce7 None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:02:05.198971 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-317b3103-9f96-479f-80cb-55989441cce7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:02:05.277994 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317b3103-9f96-479f-80cb-55989441cce7 None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:02:05.283546 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468c796d-534f-492a-92fb-f9545445f809 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abe9d8c9291845dfa4ef55034e5c9e32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:02:05.284186 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468c796d-534f-492a-92fb-f9545445f809 service placement] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:02:05.284461 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468c796d-534f-492a-92fb-f9545445f809 service placement] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:02:05.284767 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468c796d-534f-492a-92fb-f9545445f809 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:02:05.286527 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-468c796d-534f-492a-92fb-f9545445f809 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:02:05.300242 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468c796d-534f-492a-92fb-f9545445f809 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:02:05.301908 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468c796d-534f-492a-92fb-f9545445f809 service placement] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:02:05.312092 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1464/2927] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 07:02:05 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:02:10.014595 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2691fc-c984-4fff-bdff-62063f741df8 service neutron] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:02:10.016180 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-0e2691fc-c984-4fff-bdff-62063f741df8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:02:10.101008 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2691fc-c984-4fff-bdff-62063f741df8 service neutron] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:02:10.106587 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca230fae-f8e6-498d-a7b6-2ba986b55db2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb6d3743340241f98cacb9e0d063298c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:02:10.107192 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca230fae-f8e6-498d-a7b6-2ba986b55db2 service nova] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:02:10.107471 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca230fae-f8e6-498d-a7b6-2ba986b55db2 service nova] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:02:10.107743 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca230fae-f8e6-498d-a7b6-2ba986b55db2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:02:10.109504 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-ca230fae-f8e6-498d-a7b6-2ba986b55db2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:02:10.122633 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca230fae-f8e6-498d-a7b6-2ba986b55db2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:02:10.124545 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca230fae-f8e6-498d-a7b6-2ba986b55db2 service nova] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:02:10.137787 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1464/2928] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 07:02:10 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:02:15.307789 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=65 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:02:16.651361 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468c796d-534f-492a-92fb-f9545445f809 service placement] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:02:16.664650 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468c796d-534f-492a-92fb-f9545445f809 service placement] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:02:16.670261 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9f8d89-24c0-4448-baef-375bf5062085 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb6d3743340241f98cacb9e0d063298c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:02:16.670858 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9f8d89-24c0-4448-baef-375bf5062085 service nova] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:02:16.671165 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9f8d89-24c0-4448-baef-375bf5062085 service nova] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:02:16.671450 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9f8d89-24c0-4448-baef-375bf5062085 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:02:16.673203 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-4d9f8d89-24c0-4448-baef-375bf5062085 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:02:16.686706 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9f8d89-24c0-4448-baef-375bf5062085 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:02:16.688612 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9f8d89-24c0-4448-baef-375bf5062085 service nova] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:02:16.702664 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1465/2929] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 07:02:16 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:02:20.133608 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=95 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:02:23.949985 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23949cc3-ac13-4108-bb37-8eb3f342f3b4 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:02:23.950585 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23949cc3-ac13-4108-bb37-8eb3f342f3b4 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:02:23.950585 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23949cc3-ac13-4108-bb37-8eb3f342f3b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:02:23.969641 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-23949cc3-ac13-4108-bb37-8eb3f342f3b4 None None] MFA Rules not processed for user `3a2cc6be0c09472a9b3d319e82d95534`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:02:23.970844 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-23949cc3-ac13-4108-bb37-8eb3f342f3b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:02:23.977729 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1465/2930] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 07:02:23 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:02:26.697817 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=6 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:02:33.964397 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=3 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:02:48.221874 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2e40f7-2350-4046-aac5-0c932a86b554 None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:02:48.222375 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2e40f7-2350-4046-aac5-0c932a86b554 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:02:48.222466 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2e40f7-2350-4046-aac5-0c932a86b554 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:02:48.280205 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-2a2e40f7-2350-4046-aac5-0c932a86b554 None None] MFA Rules not processed for user `b82726a12436435cb931b986ef884c2e`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:02:48.281354 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-2a2e40f7-2350-4046-aac5-0c932a86b554 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:02:48.347227 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1466/2931] 10.222.0.62 () {58 vars in 1009 bytes} [Wed Aug 24 07:02:48 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:02:48.563353 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23949cc3-ac13-4108-bb37-8eb3f342f3b4 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:02:48.564794 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-23949cc3-ac13-4108-bb37-8eb3f342f3b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:02:48.659294 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23949cc3-ac13-4108-bb37-8eb3f342f3b4 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:02:48.664443 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7968a95b-3075-451f-8a4c-c27b69c3516d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14d4fed04fb144d08918bd5d4f808d23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:02:48.665086 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7968a95b-3075-451f-8a4c-c27b69c3516d service swift] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:02:48.665363 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7968a95b-3075-451f-8a4c-c27b69c3516d service swift] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:02:48.665634 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7968a95b-3075-451f-8a4c-c27b69c3516d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:02:48.674946 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7968a95b-3075-451f-8a4c-c27b69c3516d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:02:48.676600 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7968a95b-3075-451f-8a4c-c27b69c3516d service swift] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:02:48.685290 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1466/2932] 10.222.0.62 () {60 vars in 1384 bytes} [Wed Aug 24 07:02:48 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 24 07:02:58.344012 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=44 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:02:58.681902 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=41 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:03:03.552030 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2e40f7-2350-4046-aac5-0c932a86b554 None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:03:03.579012 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2e40f7-2350-4046-aac5-0c932a86b554 None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:03:03.594974 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdcf865-4186-41e6-af43-a5a49be2980a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb6d3743340241f98cacb9e0d063298c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:03:03.596012 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdcf865-4186-41e6-af43-a5a49be2980a service nova] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:03:03.598100 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdcf865-4186-41e6-af43-a5a49be2980a service nova] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:03:03.598615 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdcf865-4186-41e6-af43-a5a49be2980a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:03:03.618483 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-9fdcf865-4186-41e6-af43-a5a49be2980a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:03:03.656262 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fdcf865-4186-41e6-af43-a5a49be2980a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:03:03.660143 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fdcf865-4186-41e6-af43-a5a49be2980a service nova] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:03:03.686830 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1467/2933] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 07:03:03 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:03:06.847432 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a96bd7-c0b8-4683-8ebb-ad57f7e3f897 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:03:06.847992 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a96bd7-c0b8-4683-8ebb-ad57f7e3f897 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:03:06.847992 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a96bd7-c0b8-4683-8ebb-ad57f7e3f897 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:03:06.868422 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-a8a96bd7-c0b8-4683-8ebb-ad57f7e3f897 None None] MFA Rules not processed for user `b82726a12436435cb931b986ef884c2e`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:03:06.869675 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-a8a96bd7-c0b8-4683-8ebb-ad57f7e3f897 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:03:06.877891 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1467/2934] 10.222.0.62 () {58 vars in 967 bytes} [Wed Aug 24 07:03:06 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:03:06.890774 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdcf865-4186-41e6-af43-a5a49be2980a service nova] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:03:06.903418 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdcf865-4186-41e6-af43-a5a49be2980a service nova] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:03:06.911830 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8700608f-4158-4799-b0c2-e1300e1a7d64 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14d4fed04fb144d08918bd5d4f808d23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:03:06.913789 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8700608f-4158-4799-b0c2-e1300e1a7d64 service swift] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:03:06.914171 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8700608f-4158-4799-b0c2-e1300e1a7d64 service swift] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:03:06.919628 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8700608f-4158-4799-b0c2-e1300e1a7d64 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:03:06.936863 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8700608f-4158-4799-b0c2-e1300e1a7d64 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:03:06.939076 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8700608f-4158-4799-b0c2-e1300e1a7d64 service swift] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:03:06.949575 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1468/2935] 10.222.0.62 () {60 vars in 1384 bytes} [Wed Aug 24 07:03:06 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 07:03:11.280339 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a96bd7-c0b8-4683-8ebb-ad57f7e3f897 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:03:11.297478 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a96bd7-c0b8-4683-8ebb-ad57f7e3f897 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:03:11.302277 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7046609-4402-4b4c-a656-8da555ff6038 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:03:11.302920 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7046609-4402-4b4c-a656-8da555ff6038 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:03:11.303217 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7046609-4402-4b4c-a656-8da555ff6038 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:03:11.303469 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7046609-4402-4b4c-a656-8da555ff6038 admin admin] PATH_INFO: `/v3/users/3a2cc6be0c09472a9b3d319e82d95534` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:03:11.304644 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7046609-4402-4b4c-a656-8da555ff6038 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3a2cc6be0c09472a9b3d319e82d95534)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:03:11.306579 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7046609-4402-4b4c-a656-8da555ff6038 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:03:11.584702 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-c7046609-4402-4b4c-a656-8da555ff6038 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3a2cc6be0c09472a9b3d319e82d95534'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:03:11.591871 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-c7046609-4402-4b4c-a656-8da555ff6038 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3a2cc6be0c09472a9b3d319e82d95534'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:03:11.592613 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-c7046609-4402-4b4c-a656-8da555ff6038 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3a2cc6be0c09472a9b3d319e82d95534'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:03:11.712594 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-c7046609-4402-4b4c-a656-8da555ff6038 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3a2cc6be0c09472a9b3d319e82d95534'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:03:11.726379 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-c7046609-4402-4b4c-a656-8da555ff6038 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3a2cc6be0c09472a9b3d319e82d95534'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:03:11.732399 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1468/2936] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Aug 24 07:03:11 2022] DELETE /identity/v3/users/3a2cc6be0c09472a9b3d319e82d95534 => generated 0 bytes in 453 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:03:12.091737 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8700608f-4158-4799-b0c2-e1300e1a7d64 service swift] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:03:12.114344 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8700608f-4158-4799-b0c2-e1300e1a7d64 service swift] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:03:12.183644 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099f889c-a199-4da0-a93c-c255c30c3787 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:03:12.184294 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099f889c-a199-4da0-a93c-c255c30c3787 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:03:12.191807 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099f889c-a199-4da0-a93c-c255c30c3787 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:03:12.191807 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099f889c-a199-4da0-a93c-c255c30c3787 admin admin] PATH_INFO: `/v3/projects/057aefc9bbcc4818ab055d3c0ddacdb5` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:03:12.191807 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099f889c-a199-4da0-a93c-c255c30c3787 admin admin] RBAC: Authorizing `identity:delete_project(project_id=057aefc9bbcc4818ab055d3c0ddacdb5)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:03:12.191807 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099f889c-a199-4da0-a93c-c255c30c3787 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:03:12.330302 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-099f889c-a199-4da0-a93c-c255c30c3787 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '057aefc9bbcc4818ab055d3c0ddacdb5', 'request_id': 'req-099f889c-a199-4da0-a93c-c255c30c3787'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:03:12.337238 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-099f889c-a199-4da0-a93c-c255c30c3787 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '057aefc9bbcc4818ab055d3c0ddacdb5', 'request_id': 'req-099f889c-a199-4da0-a93c-c255c30c3787'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:03:12.345066 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-099f889c-a199-4da0-a93c-c255c30c3787 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '057aefc9bbcc4818ab055d3c0ddacdb5', 'request_id': 'req-099f889c-a199-4da0-a93c-c255c30c3787'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:03:12.502711 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-099f889c-a199-4da0-a93c-c255c30c3787 admin admin] The token cache is being invalidate because project 057aefc9bbcc4818ab055d3c0ddacdb5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81851) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 07:03:12.503047 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-099f889c-a199-4da0-a93c-c255c30c3787 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:03:12.503992 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-099f889c-a199-4da0-a93c-c255c30c3787 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:03:12.505899 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1469/2937] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Aug 24 07:03:12 2022] DELETE /identity/v3/projects/057aefc9bbcc4818ab055d3c0ddacdb5 => generated 0 bytes in 414 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:03:12.510641 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155e4b1c-4d07-4fed-852b-8799f5b7ccc3 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:03:12.511241 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155e4b1c-4d07-4fed-852b-8799f5b7ccc3 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:03:12.511425 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155e4b1c-4d07-4fed-852b-8799f5b7ccc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:03:12.537058 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-155e4b1c-4d07-4fed-852b-8799f5b7ccc3 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:03:12.538443 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-155e4b1c-4d07-4fed-852b-8799f5b7ccc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:03:12.607333 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1469/2938] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 07:03:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:03:12.615947 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5d041b-b5ad-41cb-802f-28810a4a11bb None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:03:12.616230 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5d041b-b5ad-41cb-802f-28810a4a11bb None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:03:12.616527 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5d041b-b5ad-41cb-802f-28810a4a11bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:03:12.640340 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-dc5d041b-b5ad-41cb-802f-28810a4a11bb None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:03:12.641620 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-dc5d041b-b5ad-41cb-802f-28810a4a11bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:03:12.650041 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1470/2939] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 07:03:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:03:12.654634 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155e4b1c-4d07-4fed-852b-8799f5b7ccc3 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:03:12.665221 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155e4b1c-4d07-4fed-852b-8799f5b7ccc3 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:03:12.670931 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd15da3e-fa81-4475-8f75-5a90dca2455f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:03:12.672089 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd15da3e-fa81-4475-8f75-5a90dca2455f admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:03:12.672366 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd15da3e-fa81-4475-8f75-5a90dca2455f admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:03:12.672702 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd15da3e-fa81-4475-8f75-5a90dca2455f admin admin] PATH_INFO: `/v3/domains` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:03:12.673540 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd15da3e-fa81-4475-8f75-5a90dca2455f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:03:12.674033 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd15da3e-fa81-4475-8f75-5a90dca2455f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:03:12.675761 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd15da3e-fa81-4475-8f75-5a90dca2455f admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:03:12.695955 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1470/2940] 10.222.0.62 () {58 vars in 1153 bytes} [Wed Aug 24 07:03:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:03:12.700889 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5d041b-b5ad-41cb-802f-28810a4a11bb None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:03:12.720297 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5d041b-b5ad-41cb-802f-28810a4a11bb None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:03:12.720297 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862582d5-6327-4264-bdd6-54715db4ea72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:03:12.720297 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862582d5-6327-4264-bdd6-54715db4ea72 admin admin] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:03:12.720297 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862582d5-6327-4264-bdd6-54715db4ea72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:03:12.720767 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862582d5-6327-4264-bdd6-54715db4ea72 admin admin] PATH_INFO: `/v3/projects` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:03:12.720767 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862582d5-6327-4264-bdd6-54715db4ea72 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:03:12.720916 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862582d5-6327-4264-bdd6-54715db4ea72 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:03:12.852767 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1471/2941] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Aug 24 07:03:12 2022] POST /identity/v3/projects => generated 369 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:03:12.858539 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd15da3e-fa81-4475-8f75-5a90dca2455f admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:03:12.868967 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd15da3e-fa81-4475-8f75-5a90dca2455f admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:03:12.926414 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:03:12.936240 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28936217-b950-4890-b10b-52ade6448604 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:03:12.938052 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28936217-b950-4890-b10b-52ade6448604 admin admin] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:03:12.938052 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28936217-b950-4890-b10b-52ade6448604 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:03:12.938993 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28936217-b950-4890-b10b-52ade6448604 admin admin] PATH_INFO: `/v3/users` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:03:12.939702 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28936217-b950-4890-b10b-52ade6448604 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:03:12.942119 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28936217-b950-4890-b10b-52ade6448604 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:03:13.107485 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1471/2942] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Aug 24 07:03:12 2022] POST /identity/v3/users => generated 285 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 07:03:13.113468 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862582d5-6327-4264-bdd6-54715db4ea72 admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:03:13.125575 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862582d5-6327-4264-bdd6-54715db4ea72 admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:03:13.130962 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4882a058-ab5e-4043-932b-83e4206d85b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:03:13.132042 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4882a058-ab5e-4043-932b-83e4206d85b6 admin admin] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:03:13.132424 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4882a058-ab5e-4043-932b-83e4206d85b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:03:13.132831 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4882a058-ab5e-4043-932b-83e4206d85b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:03:13.133621 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4882a058-ab5e-4043-932b-83e4206d85b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:03:13.135418 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4882a058-ab5e-4043-932b-83e4206d85b6 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:03:13.152561 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1472/2943] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Aug 24 07:03:13 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:03:13.156078 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28936217-b950-4890-b10b-52ade6448604 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:03:13.189965 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28936217-b950-4890-b10b-52ade6448604 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:03:13.189965 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765961e0-51dd-47a2-9797-8ab2d60bbe8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:03:13.189965 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765961e0-51dd-47a2-9797-8ab2d60bbe8e admin admin] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:03:13.189965 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765961e0-51dd-47a2-9797-8ab2d60bbe8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:03:13.190634 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765961e0-51dd-47a2-9797-8ab2d60bbe8e admin admin] PATH_INFO: `/v3/roles` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:03:13.190634 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765961e0-51dd-47a2-9797-8ab2d60bbe8e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:03:13.190634 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765961e0-51dd-47a2-9797-8ab2d60bbe8e admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:03:13.212427 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1472/2944] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Aug 24 07:03:13 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:03:13.217481 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4882a058-ab5e-4043-932b-83e4206d85b6 admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:03:13.233289 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4882a058-ab5e-4043-932b-83e4206d85b6 admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:03:13.238998 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df23374-ce48-4296-bbaf-9bf226a26d6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:03:13.240463 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df23374-ce48-4296-bbaf-9bf226a26d6b admin admin] REQUEST_METHOD: `PUT` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:03:13.241007 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df23374-ce48-4296-bbaf-9bf226a26d6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:03:13.241447 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df23374-ce48-4296-bbaf-9bf226a26d6b admin admin] PATH_INFO: `/v3/projects/93deec1a196b4e219a780664dbfe5c23/users/a113b24c124f424796082715fb756e27/roles/580aa7dbb38d4cff8f9d1bb8983f99ef` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:03:13.261870 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df23374-ce48-4296-bbaf-9bf226a26d6b admin admin] RBAC: Authorizing `identity:create_grant(project_id=93deec1a196b4e219a780664dbfe5c23, user_id=a113b24c124f424796082715fb756e27, role_id=580aa7dbb38d4cff8f9d1bb8983f99ef)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:03:13.264762 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df23374-ce48-4296-bbaf-9bf226a26d6b admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:03:13.462932 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:03:13.468680 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1473/2945] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Aug 24 07:03:13 2022] PUT /identity/v3/projects/93deec1a196b4e219a780664dbfe5c23/users/a113b24c124f424796082715fb756e27/roles/580aa7dbb38d4cff8f9d1bb8983f99ef => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:03:13.479833 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f03a30-6d2b-495b-afb3-d49008d0cc10 None None] REQUEST_METHOD: `POST` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:03:13.480162 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f03a30-6d2b-495b-afb3-d49008d0cc10 None None] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:03:13.480444 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f03a30-6d2b-495b-afb3-d49008d0cc10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:03:13.515585 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.auth.core [None req-b7f03a30-6d2b-495b-afb3-d49008d0cc10 None None] MFA Rules not processed for user `a113b24c124f424796082715fb756e27`. Rule list: `[]` (Enabled: `True`). {{(pid=81852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:03:13.518118 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-b7f03a30-6d2b-495b-afb3-d49008d0cc10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:03:13.604582 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1473/2946] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 07:03:13 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 07:03:13.617544 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df23374-ce48-4296-bbaf-9bf226a26d6b admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:03:13.619836 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-9df23374-ce48-4296-bbaf-9bf226a26d6b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:03:13.729970 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df23374-ce48-4296-bbaf-9bf226a26d6b admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:03:13.734382 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d52230-8f65-483e-9bf3-e07773a3dda9 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7d52162cbdc4218b577ecc1e69a88c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:03:13.735153 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d52230-8f65-483e-9bf3-e07773a3dda9 service glance] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:03:13.735500 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d52230-8f65-483e-9bf3-e07773a3dda9 service glance] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:03:13.735848 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d52230-8f65-483e-9bf3-e07773a3dda9 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:03:13.747730 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d52230-8f65-483e-9bf3-e07773a3dda9 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:03:13.749871 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d52230-8f65-483e-9bf3-e07773a3dda9 service glance] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:03:13.764679 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1474/2947] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 07:03:13 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:03:21.345738 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f03a30-6d2b-495b-afb3-d49008d0cc10 None None] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:03:21.347284 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-b7f03a30-6d2b-495b-afb3-d49008d0cc10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:03:21.419844 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f03a30-6d2b-495b-afb3-d49008d0cc10 None None] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:03:21.424425 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30170f9-bce5-49b2-bc6d-22e16b26ab2a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bb6c79db75e4d169331c02133d1ec32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:03:21.425077 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30170f9-bce5-49b2-bc6d-22e16b26ab2a service neutron] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:03:21.425486 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30170f9-bce5-49b2-bc6d-22e16b26ab2a service neutron] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:03:21.425799 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30170f9-bce5-49b2-bc6d-22e16b26ab2a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:03:21.427709 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-f30170f9-bce5-49b2-bc6d-22e16b26ab2a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:03:21.518417 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30170f9-bce5-49b2-bc6d-22e16b26ab2a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:03:21.520038 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30170f9-bce5-49b2-bc6d-22e16b26ab2a service neutron] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:03:21.533565 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1474/2948] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 07:03:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 07:03:24.101389 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=35 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:03:26.237251 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d52230-8f65-483e-9bf3-e07773a3dda9 service glance] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:03:26.239575 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-c8d52230-8f65-483e-9bf3-e07773a3dda9 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:03:26.348217 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d52230-8f65-483e-9bf3-e07773a3dda9 service glance] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:03:26.353700 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf566c0-915c-4c56-8aa9-c20275c89f45 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abe9d8c9291845dfa4ef55034e5c9e32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:03:26.354517 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf566c0-915c-4c56-8aa9-c20275c89f45 service placement] REQUEST_METHOD: `GET` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:03:26.354654 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf566c0-915c-4c56-8aa9-c20275c89f45 service placement] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:03:26.354918 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf566c0-915c-4c56-8aa9-c20275c89f45 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:03:26.356844 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-6cf566c0-915c-4c56-8aa9-c20275c89f45 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:03:26.371201 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf566c0-915c-4c56-8aa9-c20275c89f45 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:03:26.373372 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf566c0-915c-4c56-8aa9-c20275c89f45 service placement] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:03:26.393127 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1475/2949] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 07:03:26 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:03:31.526811 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=92,keystone:INSERT=3 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:03:31.588740 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30170f9-bce5-49b2-bc6d-22e16b26ab2a service neutron] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:03:31.590397 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-f30170f9-bce5-49b2-bc6d-22e16b26ab2a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:03:31.608998 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30170f9-bce5-49b2-bc6d-22e16b26ab2a service neutron] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:03:31.614284 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa5f3cb-7701-4ba8-954d-6b4abbad66d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:03:31.614941 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa5f3cb-7701-4ba8-954d-6b4abbad66d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:03:31.615440 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa5f3cb-7701-4ba8-954d-6b4abbad66d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:03:31.615751 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa5f3cb-7701-4ba8-954d-6b4abbad66d8 admin admin] PATH_INFO: `/v3/users/86ff51618d4b4b71bfad73467ddc8585` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:03:31.617012 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa5f3cb-7701-4ba8-954d-6b4abbad66d8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=86ff51618d4b4b71bfad73467ddc8585)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:03:31.618957 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa5f3cb-7701-4ba8-954d-6b4abbad66d8 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:03:31.892015 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-0fa5f3cb-7701-4ba8-954d-6b4abbad66d8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '86ff51618d4b4b71bfad73467ddc8585'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:03:31.898731 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-0fa5f3cb-7701-4ba8-954d-6b4abbad66d8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '86ff51618d4b4b71bfad73467ddc8585'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:03:31.899415 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-0fa5f3cb-7701-4ba8-954d-6b4abbad66d8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '86ff51618d4b4b71bfad73467ddc8585'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:03:32.016044 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-0fa5f3cb-7701-4ba8-954d-6b4abbad66d8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '86ff51618d4b4b71bfad73467ddc8585'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:03:32.029922 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-0fa5f3cb-7701-4ba8-954d-6b4abbad66d8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '86ff51618d4b4b71bfad73467ddc8585'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:03:32.044082 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1475/2950] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Aug 24 07:03:31 2022] DELETE /identity/v3/users/86ff51618d4b4b71bfad73467ddc8585 => generated 0 bytes in 449 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 07:03:32.347723 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf566c0-915c-4c56-8aa9-c20275c89f45 service placement] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:03:32.364132 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf566c0-915c-4c56-8aa9-c20275c89f45 service placement] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:03:32.430920 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2664b0a-970f-4154-b6e0-bc9a94d3bdf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:03:32.431576 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2664b0a-970f-4154-b6e0-bc9a94d3bdf5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:03:32.431930 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2664b0a-970f-4154-b6e0-bc9a94d3bdf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:03:32.432261 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2664b0a-970f-4154-b6e0-bc9a94d3bdf5 admin admin] PATH_INFO: `/v3/projects/01a59a0e0c9e4e8fa1c0e147e720eccb` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:03:32.433660 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2664b0a-970f-4154-b6e0-bc9a94d3bdf5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=01a59a0e0c9e4e8fa1c0e147e720eccb)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:03:32.436185 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2664b0a-970f-4154-b6e0-bc9a94d3bdf5 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:03:32.593683 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-a2664b0a-970f-4154-b6e0-bc9a94d3bdf5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '01a59a0e0c9e4e8fa1c0e147e720eccb', 'request_id': 'req-a2664b0a-970f-4154-b6e0-bc9a94d3bdf5'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:03:32.600127 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-a2664b0a-970f-4154-b6e0-bc9a94d3bdf5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '01a59a0e0c9e4e8fa1c0e147e720eccb', 'request_id': 'req-a2664b0a-970f-4154-b6e0-bc9a94d3bdf5'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:03:32.607488 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-a2664b0a-970f-4154-b6e0-bc9a94d3bdf5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '01a59a0e0c9e4e8fa1c0e147e720eccb', 'request_id': 'req-a2664b0a-970f-4154-b6e0-bc9a94d3bdf5'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:03:32.811928 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-a2664b0a-970f-4154-b6e0-bc9a94d3bdf5 admin admin] The token cache is being invalidate because project 01a59a0e0c9e4e8fa1c0e147e720eccb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81851) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 07:03:32.812442 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-a2664b0a-970f-4154-b6e0-bc9a94d3bdf5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:03:32.813671 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-a2664b0a-970f-4154-b6e0-bc9a94d3bdf5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:03:32.816255 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1476/2951] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Aug 24 07:03:32 2022] DELETE /identity/v3/projects/01a59a0e0c9e4e8fa1c0e147e720eccb => generated 0 bytes in 470 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:03:32.820642 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa5f3cb-7701-4ba8-954d-6b4abbad66d8 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:03:32.826715 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-0fa5f3cb-7701-4ba8-954d-6b4abbad66d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:03:32.838693 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa5f3cb-7701-4ba8-954d-6b4abbad66d8 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:03:32.920156 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a8fe29-2af2-4ec5-a9f6-0d5211395922 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:03:32.920942 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a8fe29-2af2-4ec5-a9f6-0d5211395922 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:03:32.921774 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a8fe29-2af2-4ec5-a9f6-0d5211395922 admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:03:32.922041 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a8fe29-2af2-4ec5-a9f6-0d5211395922 admin admin] PATH_INFO: `/v3/users/6c1ce827952f4339bf117b9e7cd431d9` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:03:32.923423 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a8fe29-2af2-4ec5-a9f6-0d5211395922 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6c1ce827952f4339bf117b9e7cd431d9)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:03:32.925611 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a8fe29-2af2-4ec5-a9f6-0d5211395922 admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:03:33.163503 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-71a8fe29-2af2-4ec5-a9f6-0d5211395922 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6c1ce827952f4339bf117b9e7cd431d9'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:03:33.171875 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-71a8fe29-2af2-4ec5-a9f6-0d5211395922 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6c1ce827952f4339bf117b9e7cd431d9'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:03:33.172825 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-71a8fe29-2af2-4ec5-a9f6-0d5211395922 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6c1ce827952f4339bf117b9e7cd431d9'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:03:33.255306 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-71a8fe29-2af2-4ec5-a9f6-0d5211395922 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6c1ce827952f4339bf117b9e7cd431d9'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:03:33.266096 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:03:33.272764 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-71a8fe29-2af2-4ec5-a9f6-0d5211395922 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6c1ce827952f4339bf117b9e7cd431d9'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:03:33.279740 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1476/2952] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Aug 24 07:03:32 2022] DELETE /identity/v3/users/6c1ce827952f4339bf117b9e7cd431d9 => generated 0 bytes in 460 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:03:33.600979 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2664b0a-970f-4154-b6e0-bc9a94d3bdf5 admin admin] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:03:33.602481 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-a2664b0a-970f-4154-b6e0-bc9a94d3bdf5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:03:33.617597 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2664b0a-970f-4154-b6e0-bc9a94d3bdf5 admin admin] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:03:33.638735 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:03:33.685938 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95759619-2d06-4afe-92dd-99d9d647c5b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:03:33.687158 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95759619-2d06-4afe-92dd-99d9d647c5b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:03:33.687448 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95759619-2d06-4afe-92dd-99d9d647c5b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:03:33.687739 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95759619-2d06-4afe-92dd-99d9d647c5b3 admin admin] PATH_INFO: `/v3/projects/17dc8467da014d3c83a2c2de396ea834` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:03:33.689088 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95759619-2d06-4afe-92dd-99d9d647c5b3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=17dc8467da014d3c83a2c2de396ea834)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:03:33.691151 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95759619-2d06-4afe-92dd-99d9d647c5b3 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:03:33.852029 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-95759619-2d06-4afe-92dd-99d9d647c5b3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '17dc8467da014d3c83a2c2de396ea834', 'request_id': 'req-95759619-2d06-4afe-92dd-99d9d647c5b3'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:03:33.859276 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-95759619-2d06-4afe-92dd-99d9d647c5b3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '17dc8467da014d3c83a2c2de396ea834', 'request_id': 'req-95759619-2d06-4afe-92dd-99d9d647c5b3'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:03:33.867563 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-95759619-2d06-4afe-92dd-99d9d647c5b3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '17dc8467da014d3c83a2c2de396ea834', 'request_id': 'req-95759619-2d06-4afe-92dd-99d9d647c5b3'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:03:34.055409 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-95759619-2d06-4afe-92dd-99d9d647c5b3 admin admin] The token cache is being invalidate because project 17dc8467da014d3c83a2c2de396ea834 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81851) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 07:03:34.055682 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-95759619-2d06-4afe-92dd-99d9d647c5b3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:03:34.056355 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-95759619-2d06-4afe-92dd-99d9d647c5b3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:03:34.058024 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1477/2953] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Aug 24 07:03:33 2022] DELETE /identity/v3/projects/17dc8467da014d3c83a2c2de396ea834 => generated 0 bytes in 458 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:03:43.398496 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=3,keystone:DELETE=1 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:03:44.079420 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=42,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:03:48.489633 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a8fe29-2af2-4ec5-a9f6-0d5211395922 admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:03:48.489633 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-71a8fe29-2af2-4ec5-a9f6-0d5211395922 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:03:48.494318 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a8fe29-2af2-4ec5-a9f6-0d5211395922 admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:03:48.548924 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb76066-12ba-4868-81f2-422531da4d4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:03:48.553097 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb76066-12ba-4868-81f2-422531da4d4a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:03:48.553097 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb76066-12ba-4868-81f2-422531da4d4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:03:48.553097 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb76066-12ba-4868-81f2-422531da4d4a admin admin] PATH_INFO: `/v3/users/a113b24c124f424796082715fb756e27` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:03:48.553097 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb76066-12ba-4868-81f2-422531da4d4a admin admin] RBAC: Authorizing `identity:delete_user(user_id=a113b24c124f424796082715fb756e27)` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:03:48.553097 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb76066-12ba-4868-81f2-422531da4d4a admin admin] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:03:48.954410 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-0cb76066-12ba-4868-81f2-422531da4d4a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a113b24c124f424796082715fb756e27'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:03:48.961487 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-0cb76066-12ba-4868-81f2-422531da4d4a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a113b24c124f424796082715fb756e27'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:03:48.962170 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-0cb76066-12ba-4868-81f2-422531da4d4a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a113b24c124f424796082715fb756e27'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:03:49.089253 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-0cb76066-12ba-4868-81f2-422531da4d4a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a113b24c124f424796082715fb756e27'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:03:49.103469 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.notifications [None req-0cb76066-12ba-4868-81f2-422531da4d4a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a113b24c124f424796082715fb756e27'} {{(pid=81852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:03:49.110104 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1477/2954] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Aug 24 07:03:48 2022] DELETE /identity/v3/users/a113b24c124f424796082715fb756e27 => generated 0 bytes in 630 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:03:49.114958 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db323517-5dd7-4ff7-8894-3837705b3270 None None] REQUEST_METHOD: `POST` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:03:49.115692 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db323517-5dd7-4ff7-8894-3837705b3270 None None] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:03:49.115692 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db323517-5dd7-4ff7-8894-3837705b3270 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:03:49.136189 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.auth.core [None req-db323517-5dd7-4ff7-8894-3837705b3270 None None] MFA Rules not processed for user `8b2c3a376af245f2a0e347e2f08885a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:03:49.137521 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-db323517-5dd7-4ff7-8894-3837705b3270 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:03:49.197011 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1478/2955] 10.222.0.62 () {58 vars in 958 bytes} [Wed Aug 24 07:03:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:03:49.207910 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb76066-12ba-4868-81f2-422531da4d4a admin admin] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:03:49.209366 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-0cb76066-12ba-4868-81f2-422531da4d4a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:03:49.266470 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb76066-12ba-4868-81f2-422531da4d4a admin admin] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:03:49.270504 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c504a2b-8f77-4da3-a321-ec09987181d8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bb6c79db75e4d169331c02133d1ec32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:03:49.271200 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c504a2b-8f77-4da3-a321-ec09987181d8 service neutron] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:03:49.271600 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c504a2b-8f77-4da3-a321-ec09987181d8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:03:49.271952 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c504a2b-8f77-4da3-a321-ec09987181d8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:03:49.281132 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c504a2b-8f77-4da3-a321-ec09987181d8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:03:49.282568 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c504a2b-8f77-4da3-a321-ec09987181d8 service neutron] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:03:49.291846 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1478/2956] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 07:03:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 07:03:49.742522 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db323517-5dd7-4ff7-8894-3837705b3270 None None] Authenticating user token {{(pid=81851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:03:49.744009 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.fernet_utils [None req-db323517-5dd7-4ff7-8894-3837705b3270 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:03:49.757987 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db323517-5dd7-4ff7-8894-3837705b3270 None None] Validating token access rules against request {{(pid=81851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:03:49.762860 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87699da-3a74-4721-b6ee-bf72253da341 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b2c3a376af245f2a0e347e2f08885a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '632e93ce840f4853b64b4d659536e045', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:03:49.763490 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87699da-3a74-4721-b6ee-bf72253da341 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:03:49.763775 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87699da-3a74-4721-b6ee-bf72253da341 admin admin] SCRIPT_NAME: `/identity` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:03:49.764315 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87699da-3a74-4721-b6ee-bf72253da341 admin admin] PATH_INFO: `/v3/projects/93deec1a196b4e219a780664dbfe5c23` {{(pid=81851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:03:49.765539 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87699da-3a74-4721-b6ee-bf72253da341 admin admin] RBAC: Authorizing `identity:delete_project(project_id=93deec1a196b4e219a780664dbfe5c23)` {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:03:49.767594 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87699da-3a74-4721-b6ee-bf72253da341 admin admin] RBAC: Authorization granted {{(pid=81851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:03:49.867299 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-f87699da-3a74-4721-b6ee-bf72253da341 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '93deec1a196b4e219a780664dbfe5c23', 'request_id': 'req-f87699da-3a74-4721-b6ee-bf72253da341'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:03:49.872890 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-f87699da-3a74-4721-b6ee-bf72253da341 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '93deec1a196b4e219a780664dbfe5c23', 'request_id': 'req-f87699da-3a74-4721-b6ee-bf72253da341'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:03:49.879979 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-f87699da-3a74-4721-b6ee-bf72253da341 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '93deec1a196b4e219a780664dbfe5c23', 'request_id': 'req-f87699da-3a74-4721-b6ee-bf72253da341'} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:03:49.975098 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-f87699da-3a74-4721-b6ee-bf72253da341 admin admin] The token cache is being invalidate because project 93deec1a196b4e219a780664dbfe5c23 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81851) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 07:03:49.975393 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-f87699da-3a74-4721-b6ee-bf72253da341 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:03:49.976056 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG keystone.notifications [None req-f87699da-3a74-4721-b6ee-bf72253da341 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:03:49.977910 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: [pid: 81851|app: 0|req: 1479/2957] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Aug 24 07:03:49 2022] DELETE /identity/v3/projects/93deec1a196b4e219a780664dbfe5c23 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:03:59.287751 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:03:59.971739 dv-cinder-854269-1-iscsi devstack@keystone.service[81851]: DEBUG dbcounter [-] [81851] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:06:19.843244 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c504a2b-8f77-4da3-a321-ec09987181d8 service neutron] Authenticating user token {{(pid=81852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:06:19.844731 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-1c504a2b-8f77-4da3-a321-ec09987181d8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:06:19.967704 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c504a2b-8f77-4da3-a321-ec09987181d8 service neutron] Validating token access rules against request {{(pid=81852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:06:20.032017 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bb1fdf-2af1-4e36-93d8-76604591e55b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abe9d8c9291845dfa4ef55034e5c9e32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '181f4762037e454489d9552cc603179c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:06:20.032643 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bb1fdf-2af1-4e36-93d8-76604591e55b service placement] REQUEST_METHOD: `GET` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:06:20.036952 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bb1fdf-2af1-4e36-93d8-76604591e55b service placement] SCRIPT_NAME: `/identity` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:06:20.036952 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bb1fdf-2af1-4e36-93d8-76604591e55b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:06:20.036952 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.fernet_utils [None req-41bb1fdf-2af1-4e36-93d8-76604591e55b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:06:20.047549 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41bb1fdf-2af1-4e36-93d8-76604591e55b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:06:20.049174 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41bb1fdf-2af1-4e36-93d8-76604591e55b service placement] RBAC: Authorization granted {{(pid=81852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:06:20.060398 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: [pid: 81852|app: 0|req: 1479/2958] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Aug 24 07:06:19 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:06:30.055477 dv-cinder-854269-1-iscsi devstack@keystone.service[81852]: DEBUG dbcounter [-] [81852] Writing DB stats keystone:SELECT=80 {{(pid=81852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}